Building object frame model automatic extraction method based on graph cut

Through the graph-based cutting method, the frame model of urban-level buildings is extracted, which solves the problems of inefficiency and global regular rules guarantee in the existing technology, and realizes efficient building model extraction and lightweight reconstruction, which is suitable for urban-level batch models. deal with.

CN118918286AActive Publication Date: 2024-11-08BEIJING INSTITUTE OF SURVEYING AND MAPPING

Patent Information

Application Number
CN202410960577.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-17
Publication Date
2024-11-08
Estimated Expiration
2044-07-17

AI Technical Summary

Technical Problem

The prior art is inefficient in the processing of geographic scenario model data at the city level and cannot guarantee global regular rules, which affects the visualization effect, especially in the simplified method of building single-unit acquisition and automation of local errors.

Method used

Using a graph-based cutting method, the building body frame model is automatically extracted through plane extraction, merging and regularization, multihedral cutting and visibility information binary marking, and the light information and visibility analysis of densely matched point clouds are constructed. The cutting model distinguishes the internal and external of the polyhedron.

Benefits of technology

It realizes efficient building frame model extraction, reduces manual workload, is suitable for city-level batch building model reconstruction, generates lightweight models, facilitates storage and rendering, and supports GIS quantitative analysis.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118918286A_ABST
    Figure CN118918286A_ABST
Patent Text Reader

Abstract

The invention discloses a graph cut-based building object frame model automatic extraction method, which comprises the following steps: firstly, extracting a plane from a point cloud, and converting building reconstruction into a graph cut problem to solve by using light information of a densely matched point cloud as prior information, thereby avoiding huge calculation amount caused by dynamic planning solution; carrying out combination and regularization optimization on the data; then, cutting an outer bounding box of the point cloud by using a plane to obtain candidate polyhedrons; and finally, constructing a graph cut model to perform binary marking on the polyhedrons by using visual information, and distinguishing the polyhedrons inside and outside the building so as to realize the extraction of the building object frame model.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the fields of three-dimensional digitization technology, point cloud processing technology, and surveying and mapping technology, and in particular to an automatic extraction method for building object frame models based on graph cut. Background Art

[0002] Three-dimensional building models have wide applications in fields such as urban planning, three-dimensional navigation, disaster management, and water conservancy and hydropower.

[0003] Currently, with the development of technologies such as low-altitude oblique photogrammetry and UAV photogrammetry, the accuracy of geographic scene model data is becoming increasingly higher. However, city-level geographic scene model data is extremely large, reaching TB levels at a scale of 5 centimeters, which is not conducive to subsequent analysis and application. In addition, geographic scenes expressed in the form of 3D mesh models lack individual attributes, making attribute linking and subsequent quantitative GIS analysis impossible. On the display end, 3D mesh models contain too many useless details, which is not conducive to storage and rendering, thus requiring lightweighting. If manual collection methods are used to collect individual building data, the efficiency is too low; while automated local error simplification methods such as QEM (quadric error metric) and Lindstorm can only simplify the model structure but cannot guarantee global regularization rules, which will greatly affect the visualization effect.

[0004] In summary, as an important component of urban real-scene 3D information products, 3D building models have wide applications in urban planning, 3D navigation, disaster management, water conservancy and hydropower, and other fields. However, manually collecting data on individual buildings is extremely inefficient; while traditional automated local error simplification methods can only simplify the model structure but cannot guarantee global regularization rules, which greatly affects the visualization effect. Summary of the Invention

[0005] The purpose of this invention is to provide an automatic extraction method for building object frame models based on graph cuts, which solves the above-mentioned technical problems pointed out in the prior art.

[0006] This invention provides a method for automatically extracting the bounding box model of a building object based on graph cuts, comprising the following steps:

[0007] Plane extraction: Acquire 3D point cloud data of the current building; perform plane extraction operation based on the 3D point cloud data to obtain multiple planes corresponding to the building;

[0008] Plane merging and regularization: Based on global regularization, all the planes are optimized and merged to obtain multiple new planes;

[0009] Polyhedral cutting: Construct a polyhedron based on all the new planes; cut the polyhedron to obtain multiple candidate planes;

[0010] Binary labeling of polyhedral graph cuts based on visibility information: Multiple second polyhedra are constructed based on each of the candidate planes; the second polyhedra are binary-labeled through visibility analysis to obtain the polyhedron inside the building surface and the polyhedron outside the building surface;

[0011] Building surface model extraction: The building model is obtained by merging the polyhedra S inside the building surface.

[0012] Preferably, the step of optimizing and merging all the planes based on global regularization to obtain multiple new planes includes the following steps:

[0013] Based on all the aforementioned planes, obtain multiple plane pairs (s) i ,s j ); calculate and obtain the plane pair (s) i ,s j The included angle θ;

[0014] Obtain each of the plane pairs (s) i ,s j The number of support points N for each plane in the equation;

[0015] Traverse each of the aforementioned plane pairs (s) i ,s j ), determine the plane pair (s) i ,s j Is the included angle θ less than or equal to the angle threshold θ? t And the plane pair (s) i ,s j Is the number of support points N for each plane in the equation greater than or equal to the point count threshold N? t If so, then the plane pair (s) i ,s j If not, merge the planes (s) to obtain a new plane; otherwise, merge the planes (s) to obtain a new plane. i ,s j The two planes are respectively used as new planes.

[0016] Preferably, the plane pair (s) i ,s j ) represents two adjacent planes.

[0017] Preferably, the step of cutting the polyhedron to obtain multiple candidate planes includes the following steps:

[0018] Calculate and obtain the bounding boxes corresponding to all the current 3D point cloud data;

[0019] The bounding box is subjected to surface extraction to obtain the surface corresponding to each new plane;

[0020] After arranging the various covering surfaces in descending order of area, the covering surfaces are traversed sequentially from largest to smallest area to determine whether the polyhedron intersects with the covering surface. If so, the covering surface is used to divide the polyhedron to obtain multiple candidate planes.

[0021] Preferably, the step of performing binary labeling on the second polyhedron through visibility analysis to obtain the internal polyhedron and the external polyhedron of the building surface includes the following steps:

[0022] Construct an empty initial directed graph; map the second polyhedron primitive corresponding to the second polyhedron to the initial directed graph to obtain vertex V, and at the same time map the common face of every two adjacent polyhedra to the initial directed graph to obtain edge E, thus obtaining the directed graph;

[0023] Add a source and a sink to each vertex V in the directed graph; obtain the weight w of each edge E;

[0024] Based on the source vertex, sink vertex, and weight w, all vertices V in the directed graph are processed by st cut to obtain subsets S and T; wherein the source vertex belongs to subset S; and the sink vertex belongs to subset T.

[0025] The internal polyhedron of the building surface is obtained based on the subset S, and the external polyhedron of the building surface is obtained based on the subset T.

[0026] Preferably, after "adding a source and a sink to each vertex V in the directed graph", the method further includes: connecting each vertex to the source and the sink respectively.

[0027] Preferably, the subset S represents the vertices of the polyhedron inside the building surface;

[0028] The subset T represents the vertices of the external polyhedron on the building surface.

[0029] Preferably, the step of obtaining subsets S and T from all vertices V in the directed graph through st cut based on the source, sink, and weight w includes: first obtaining subset S by minimizing global energy, and then obtaining subset T consisting of all vertices other than subset S.

[0030] Preferably, the step of obtaining the subset S by minimizing global energy includes the following steps:

[0031] Traverse each of the three-dimensional point cloud data: Obtain the visibility item E of each of the three-dimensional point cloud data. vis (S) and obtain the directional term E of each of the three-dimensional point cloud data. orient (S);

[0032] Based on the aforementioned visibility item E vis (S) and directional term E orient (S) Calculate and obtain the energy function E(S);

[0033] Determine whether the energy function E(S) is greater than 0; if so, determine that the three-dimensional point cloud data is the three-dimensional point cloud data of the external polyhedron of the building surface; construct a subset S based on the three-dimensional point cloud data of all the external polyhedrons of the building surface.

[0034] Preferably, the energy function E(S) is calculated as follows:

[0035] E(S)=E vis (S)+E orient (S).

[0036] Compared with the prior art, the embodiments of the present invention have at least the following technical advantages:

[0037] Analysis of the above-mentioned automatic extraction method for building object bounding box models based on graph cut provided by this invention reveals that, in practical applications, planes are first extracted from point clouds. The ray information of densely matched point clouds is used as prior information to transform building reconstruction into a graph cut problem, avoiding the huge computational load caused by dynamic programming. These planes are then merged and regularized for optimization. Next, the bounding boxes of the point clouds are cut using planes to obtain candidate polyhedra. Finally, using visibility information, a graph cut model is constructed to binary-label the polyhedra, distinguishing between polyhedra inside and outside the building, thus achieving the extraction of the building bounding box model. Experimental results verify that the algorithm of this application can achieve good results on noisy point clouds of buildings. Furthermore, this application achieves GPU parallelization of visibility weight calculation, thereby achieving the goal of efficiently generating building bounding box models. This makes the method suitable for batch building model reconstruction at the city level; it automatically generates lightweight single-building models, which are beneficial for storage, rendering, and subsequent GIS quantitative analysis, greatly reducing manual workload. Attached Figure Description

[0038] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0039] Figure 1 A schematic diagram illustrating the overall operation steps of an automatic extraction method for building object frame models based on graph cuts provided by this invention;

[0040] Figure 2 A schematic diagram of the original point cloud for an automatic extraction method of building object frame model based on graph cut provided by the present invention;

[0041] Figure 3 This is a schematic diagram of the planar extraction result of an automatic extraction method for building object frame models based on graph cut provided by the present invention;

[0042] Figure 4 This is a schematic diagram of the result of plane merging in an automatic extraction method for building object frame models based on graph cut provided by the present invention.

[0043] Figure 5 The building reconstruction result provided by the automatic extraction method of building object frame model based on graph cut provided by this invention Figure 1 Simulation diagram;

[0044] Figure 6 The building reconstruction result provided by the automatic extraction method of building object frame model based on graph cut provided by this invention Figure 2 Simulation diagram;

[0045] Figure 7 A schematic diagram illustrating the steps for obtaining multiple new planes in an automatic extraction method for building object frame models based on graph cut provided by the present invention.

[0046] Figure 8 A schematic diagram simulating the effect of planar regularization in an automatic extraction method for building object frame models based on graph cut provided by the present invention;

[0047] Figure 9 An unregularized rendering of a method for automatically extracting building object frame models based on graph cuts provided by this invention;

[0048] Figure 10 A schematic diagram illustrating the steps of obtaining multiple candidate planes in an automatic extraction method for building object frame models based on graph cut provided by the present invention.

[0049] Figure 11A bounding box and planar simulation diagram of an automatic extraction method for building object frame models based on graph cut provided by the present invention;

[0050] Figure 12 This is a schematic diagram of a planar sorting simulation of an automatic extraction method for building object frame models based on graph cuts provided by the present invention;

[0051] Figure 13 A schematic diagram illustrating the construction of a polyhedron simulation based on a method for automatically extracting building object frame models using graph cuts, provided by this invention.

[0052] Figure 14 A schematic diagram illustrating the operation steps of an automatic extraction method for building object frame models based on graph cut, which provides the present invention, to obtain the internal polyhedrons and external polyhedrons of the building surface;

[0053] Figure 15 A schematic diagram illustrating the steps for obtaining a subset S using an automatic extraction method for building object frame models based on graph cuts provided by this invention.

[0054] Figure 16 This is a visualization calculation diagram of an automatic extraction method for building object frame models based on graph cuts provided by the present invention. Detailed Implementation

[0055] The technical solution of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0056] The present invention will now be described in further detail with reference to specific embodiments and accompanying drawings.

[0057] like Figure 1 As shown, this invention proposes an automatic extraction method for building object bounding box models based on graph cuts, including the following steps:

[0058] Step S10: Plane Extraction: Acquire the 3D point cloud data of the current building; perform plane extraction operation based on the 3D point cloud data to obtain multiple planes corresponding to the building;

[0059] It should be noted that the embodiments of this application generate dense point clouds (i.e., the aforementioned three-dimensional point cloud data) by capturing low-altitude oblique images. Figure 2(As shown); The planar extraction method used in this embodiment optimizes the continuous planar parameters and the discrete allocation of input points (i.e., the aforementioned 3D point cloud data) by starting from an initial configuration and seeking high fidelity, high simplicity, and high integrity; it utilizes an exploration mechanism guided by a multi-objective energy function and employs five geometric operations to handle transformations within the solution space; the vast solution space in its multi-objective energy function-guided exploration mechanism is processed by five geometric operators to create, delete, and modify primitives; its multi-objective energy function is as follows:

[0060] U(x)=ω f U f (x)+ω s U s (x)+ω c U c (x);

[0061] Among them U f (x), U s (x) and U c (x) represent the fidelity, simplicity, and completeness terms, respectively, and their ranges are all [0,1]; ω f ω s and ω c Each of these has a corresponding weight, all of which are positive and their sum is 1.

[0062] By balancing these three objectives through the aforementioned multi-objective energy function, a high-quality planar primitive configuration can be found. By minimizing this energy function, the algorithm can improve the initial configuration to achieve higher fidelity, simplicity, and completeness.

[0063] The aforementioned planar extraction method and the aforementioned multi-objective energy function are existing technologies and will not be described in detail here.

[0064] Because the point cloud obtained by dense matching has high noise, the extracted planes inevitably have some errors and irregularities, which may lead to errors in subsequent reconstruction. To mitigate the impact of errors in plane extraction, this embodiment of the application merges and regularizes the planes in subsequent operations to obtain new planes (such as...). Figure 3 (as shown), thereby reducing point cloud noise;

[0065] Step S20: Plane merging and regularization: Based on global regularization, all the planes are optimized and merged to obtain multiple new planes;

[0066] It should be noted that due to the presence of noise and outliers, planar extraction may generate some unwanted planar segments; these segments may cause elongated structures and small protrusions in the final model; in addition, they also lead to additional computational costs and reduce efficiency. To address this issue, embodiments of this application employ a planar iterative merging optimization method, which effectively reduces point cloud noise and improves the accuracy of planar extraction (e.g., Figure 4 (as shown);

[0067] Step S30: Polyhedron cutting: Construct a polyhedron based on all the new planes; cut the polyhedron to obtain multiple candidate planes;

[0068] It should be noted that after obtaining new planes through plane merging and regularization, a polyhedron of a spatially complex building will be constructed using these new planes. In order to further identify the building model, this polyhedron needs to be further processed to obtain the plane of each face of the polyhedron (i.e., the above candidate planes).

[0069] When further processing the polyhedron to obtain candidate planes, we can exhaustively enumerate all the polyhedron primitives (polyhedron primitives are polyhedron objects, which are the components of a polyhedron and can be points, lines, or faces) cut out by the supporting planes in the entire scene. However, this method has a very high space complexity, reaching Θ(n 3 The computational load is at the level of ) level;

[0070] In fact, each additional planar primitive (adding a new planar primitive) will split all the current polyhedral elements onto its supporting plane (the supporting plane refers to the fact that a polyhedron is composed of multiple planes, and after the new plane intersects with the polyhedron, the new plane will become part of the new polyhedron); in addition, not considering the spatial extent of the planar primitive may generate some meaningless polyhedral primitives, for example, a chimney on the roof may interfere with the segmentation of polyhedral primitives on the other side of the building; considering the computational load and memory cost, the embodiments of this application use a more refined segmentation method, the specific operation of which is shown in steps S31-S33.

[0071] Step S40: Binary labeling of polyhedral graph cut based on visibility information: Multiple second polyhedra are constructed based on each of the candidate planes; the second polyhedra are binary labeled through visibility analysis to obtain the building surface interior polyhedron S and the building surface exterior polyhedron;

[0072] It should be noted that the three-dimensional point cloud data of the building surface is generated by photographing the building. In this three-dimensional point cloud data, based on the visibility of light as a constraint, the polyhedrons of the building exterior are represented. The polyhedrons of the building exterior are visible, while the polyhedrons of the building interior are not visible. Therefore, the generated polyhedrons of the building surface will be divided into two categories: exterior and interior (i.e., exterior and interior of the building).

[0073] Based on this, the embodiments of this application transform the building surface reconstruction problem into a binary classification problem of polyhedra. The visibility of the polyhedra is analyzed, and each polyhedra is marked as belonging to the interior or exterior of the building surface. Then, the interior polyhedra are merged to obtain a watertight building model without self-intersections.

[0074] The aforementioned visibility information refers to the fact that during the generation stage of dense point clouds (i.e., the aforementioned 3D point cloud data), each 3D point cloud data can obtain image ray information, i.e., visibility information; the embodiments of this application utilize this visibility to construct a visual item E. vis (S), which increases the weight of the surface traversed by the line of sight (the weight is the aforementioned visibility term E). vis The principle is that if a point in the point cloud is on the surface of a building, then its corresponding line of sight, that is, the line formed by the point and the photographic center of its visible image, should be unobstructed; therefore, the weight of the edge can be assigned by the intersection of the line of sight and the polyhedron.

[0075] Step S50: Building surface model extraction: The building model is obtained by merging the polyhedra S inside the building surface.

[0076] It should be noted that the above embodiments of this application use the maximum flow minimum cut algorithm to solve the problem. After obtaining the category of each polyhedron, the polyhedra S inside the building surface are merged to obtain a watertight building model without self-intersections.

[0077] For example, the embodiments of this application have been fully verified in actual experiments and applied in the automatic generation project of building LOD models. In addition, such as... Figure 5 and Figure 6 As shown, this application uses four dense point clouds of buildings with different complexities for experiments, with the number of points ranging from 200,000 to 1 million. The point clouds in this dataset are all derived from dense matching of oblique images, thus containing a large proportion of outliers and noise.

[0078] Figure 5 For the results of building reconstruction Figure 1 and Figure 6 For the results of building reconstruction Figure 2The images show buildings reconstructed by the method of this application and the Polyfit algorithm. Table 1 shows the time consumption statistics for reconstructing buildings of different complexities using Polyfit, CPU, and GPU. It can be seen that the method of this application obtains lightweight reconstruction models of these buildings well, highlighting their main planar features and preserving clear edges. Polyfit is suitable for buildings with simple structures, but encounters computational bottlenecks for buildings with complex structures. Compared with Polyfit, the algorithm of this application can handle complex buildings, such as buildings (c) and (d), which have 74 and 91 faces, respectively. Polyfit is more efficient on simple buildings (a) and (e) because at this order of magnitude of candidate primitives, the advantage of graph cut does not far outweigh that of dynamic programming, while the cutting of 3D candidate polyhedra and computation of visibility in this application takes longer than the time required by Polyfit to calculate candidate faces. However, for more complex buildings (b), (f), and (g), the time required by Polyfit is 5 to more than 100 times that of the algorithm of this application, and for (c), (d), and (h), it is completely unsolvable. The solution speed advantage of graph cut is very obvious in these structures.

[0079] Table 1. Experimental data parameters and Polyfit, CPU, and GPU time consumption statistics (unit: seconds)

[0080]

[0081] Table 2 uses buildings (a)-(d) as examples to statistically analyze the four main steps: planar processing, polyhedron construction, visibility calculation, and graph cutting. It can be seen that visibility calculation is the most time-consuming step, accounting for two-thirds of the total time, while graph cutting itself consumes very little time. It is worth noting that on datasets with millions of points, the maximum computation time of the method in this embodiment does not exceed 15 minutes. Using GPU parallel computing can improve the speed of visibility calculation by 5 to 20 times, and the improvement becomes more significant as the complexity of the building increases.

[0082] Table 2. Time Statistics for Main Steps (Unit: seconds)

[0083]

[0084] Based on the above qualitative and quantitative experimental analysis, it can be concluded that the building reconstruction algorithm based on visibility and graph cut proposed in this application can reconstruct urban buildings with high accuracy and efficiency, especially for relatively complex buildings, it has high efficiency.

[0085] In summary, this application first extracts planes from point clouds, utilizing the ray information of densely matched point clouds as prior information to transform building reconstruction into a graph cut problem, avoiding the huge computational load caused by dynamic programming. These planes are then merged and regularized for optimization. Next, the bounding boxes of the point clouds are cut using planes to obtain candidate polyhedra. Finally, using visibility information, a graph cut model is constructed to binary-label the polyhedra, distinguishing between polyhedra inside and outside the building, thus achieving building frame model extraction. Experimental results verify that the algorithm of this application can achieve good results on noisy point clouds of buildings. Furthermore, this application achieves GPU parallelization of visibility weight calculation, thereby achieving the goal of efficiently generating building frame models, making this method applicable to batch building model reconstruction at the city level. It automatically generates lightweight single-building models, facilitating storage, rendering, and subsequent GIS quantitative analysis, significantly reducing manual workload.

[0086] Specifically, if Figure 7 As shown, in step S20, the optimization and merging of all the planes based on global regularization to obtain multiple new planes includes the following steps:

[0087] Step S21: Obtain multiple plane pairs (s) based on all the planes. i ,s j ); calculate and obtain the plane pair (s) i ,s j The included angle θ;

[0088] The plane pair (s) i ,s j ) represents two adjacent planes; that is, the above s i With the above s j Let and represent a plane, and plane s i With plane s j Adjacent;

[0089] The above calculation yields the plane pair (s) i ,s j The included angle θ is obtained by calculating the included angle between the normal vectors of the two planes, which will not be elaborated in this application;

[0090] Step S22: Obtain each of the plane pairs (s) i ,s j The number of support points N for each plane in the equation;

[0091] Step S23: Traverse each of the plane pairs (s i ,s j ), determine the plane pair (s) i ,s jIs the included angle θ less than or equal to the angle threshold θ? t And the plane pair (s) i ,s j Is the number of support points N for each plane in the equation greater than or equal to the point count threshold N? t If (i.e., simultaneously satisfying the above judgment conditions, i.e., plane pair (s) i ,s j The included angle θ is less than or equal to the angle threshold θ. t And satisfy the plane pair (s) i ,s j The number of support points N for each plane in the equation is greater than or equal to the point count threshold N. t ), then the plane pair (s) i ,s j If not, merge the planes (s) to obtain a new plane; otherwise, merge the planes (s) to obtain a new plane. i ,s j The two planes are respectively used as new planes (i.e., plane s). i As a new plane, at the same time s j (as a new plane);

[0092] Repeat the above operation until all the plane pairs (s) have been traversed. i ,s j This results in multiple new planes;

[0093] It should be noted that the above embodiments of this application are based on first calculating each pair of planes (s) i ,s j (each pair of planes (s) i ,s j That is, the aforementioned plane pair (s) i ,s j The angle θ between the planes is calculated; then, starting with the plane pair with the smallest angle, each plane pair (s) is traversed. i ,s j ), test whether the following two conditions are met: (1) plane pair (s i ,s j The angle θ between the two planes in the given information is less than the threshold θ. t (i.e., the aforementioned plane pair (s) i ,s j The included angle θ is less than or equal to the angle threshold θ. t (2) Exceeding the specified threshold N t The point lies on two planes (i.e., the aforementioned plane pair (s) i ,s j The number of support points N for each plane in the equation is greater than or equal to the point count threshold N. t );

[0094] If both conditions are met, the two planes are merged to obtain a new plane (the point clouds on the two planes are merged into the same plane, i.e., the plane is formed by the combination of (s) i ,s j Combine all the 3D point cloud data on the plane to obtain a new plane, and use its corresponding points to generate a new plane; repeat this process until there are no more plane pairs that can be merged.

[0095] The experiment in this embodiment selects an angle threshold θ. t The degree is 10, and the point threshold is N. t =min(|s i |,|s j |) / 5, where |s i | represents plane s i The number of support points (the number of support points refers to the number of points contained in the plane);

[0096] like Figure 3 He Ru Figure 4 The difference between the planes before and after the merger is shown, and the number of planes also decreased from 46 to 15;

[0097] This application's embodiments perform global regularization on planes, changing their orientation and position to regulate their geometric relationships, namely parallelism, coplanarity, and orthogonality. The main idea is to first organize the geometric relationships between planes in layers (i.e., the included angles between each pair of planes mentioned above), and then use a greedy algorithm to normalize the planes (i.e., merging the planes mentioned above). After adjustment, planes that are nearly parallel and nearly orthogonal are respectively adjusted to be completely coplanar and orthogonal. The effect of plane regularization is as follows: Figure 8 As shown, Figure 9 The image shows the effect before regularization; it can be seen that it significantly improves the structural regularity and reduces the complexity of the reconstructed building structure.

[0098] Specifically, if Figure 10 As shown, in step S30, the polyhedron is cut to obtain multiple candidate planes, including the following steps:

[0099] Step S31: Calculate and obtain the bounding boxes corresponding to all the current 3D point cloud data;

[0100] It should be noted that if Figure 2 As shown, the bounding box mentioned above serves as the starting point for subsequent polyhedral segmentation. This is because, theoretically, a plane is boundless. When space is cut by a plane, in order to close the space, a three-dimensional space (bounding box) is given for cutting. This bounding box is the point cloud bounding box.

[0101] Step S32: Extract the coverage surface of the bounding box to obtain the coverage surface corresponding to each new plane;

[0102] It should be noted that the above-described overlay extraction operation utilizes 2Dα-shape, a mathematical method used to extract closed convex polygons from a point set. These polygons can approximate the shape features of the original point cloud data. Here, 2Dα-shape is used to extract the overlay (or overlay primitive) of each plane, that is, to extract the boundary or contour of the plane, such as... Figure 11 As shown;

[0103] Step S33: Arrange each of the covering surfaces in descending order of area, and then traverse each covering surface in descending order of area to determine whether the polyhedron intersects with the covering surface. If so, use the covering surface to divide the polyhedron to obtain multiple candidate planes.

[0104] It should be noted that in the above embodiments of this application, the surface primitives (i.e., the aforementioned covering surface primitives or covering surfaces) are sorted and traversed (e.g., ...). Figure 12 As shown, if a polyhedron intersects with a face primitive, then the polyhedron is divided to obtain the final result. Figure 13 The polyhedron shown is an example of a polyhedron. It is worth noting that the final segmentation result may be affected by the order in which the face primitives are cut. In this embodiment, the face primitives are arranged in descending order of area because larger ones are more stable. Cutting with stable faces first can make the segmentation result more stable.

[0105] Specifically, if Figure 14 As shown, in step S40, the second polyhedron is binary-labeled through visibility analysis to obtain the polyhedron inside the building surface and the polyhedron outside the building surface, including the following operation steps:

[0106] Step S41: Construct an empty initial directed graph; map the second polyhedron primitive corresponding to the second polyhedron to the initial directed graph to obtain vertex V, and at the same time map the common face of every two adjacent polyhedra to the initial directed graph to obtain edge E, thus obtaining the directed graph;

[0107] Step S42: Add a source and a sink to each vertex V in the directed graph; obtain the weight w of each edge E;

[0108] After “adding a source and a sink to each vertex V in the directed graph”, the method further includes: connecting each vertex to the source and the sink respectively.

[0109] The weight w ≥ 0;

[0110] It should be noted that the above embodiments of this application regard this binary classification problem (i.e., the above-mentioned problem of reconstructing the building surface is transformed into the binary classification problem of the obtained polyhedron) as the st cut problem of the directed graph G = (V, E); by first constructing a directed graph: mapping the polyhedron primitives to the vertices (V) of the graph, and mapping the face shared by two adjacent polyhedra to the edges (E) of the graph.

[0111] The above embodiment of this application augments vertex V by adding a source vertex and a sink vertex; each vertex is connected to both the source and sink vertex; all edges have non-negative weights w.

[0112] The source point can be selected from a certain internal area of ​​the building, usually a representative point inside the building surface; the sink point is selected from a point outside the building, usually a point far away from the building (i.e., the source point is a point of the polyhedron inside the building surface; the sink point is a point of the polyhedron outside the building surface).

[0113] Step S43: Based on the source vertex, sink vertex, and weight w, all vertices V in the directed graph are processed by st cut to obtain subsets S and T; wherein, the source vertex belongs to subset S; and the sink vertex belongs to subset T.

[0114] Wherein, the subset S represents the vertices of the polyhedron inside the building surface;

[0115] The subset T represents the vertices in the external polyhedron of the building surface;

[0116] The step of obtaining subsets S and T from all vertices V in the directed graph through st cut based on the source, sink and weight w includes: first obtaining subset S by distinguishing them through global energy minimization, and then obtaining subset T consisting of all vertices other than subset S;

[0117] Step S44: Obtain the internal polyhedron of the building surface based on the subset S, and obtain the external polyhedron of the building surface based on the subset T;

[0118] In the above embodiment of this application, after distinguishing the subsets to which each vertex of the second polyhedron belongs, the second polyhedron corresponding to the vertex belonging to subset S is identified as the internal polyhedron of the building surface, and the second polyhedron corresponding to the vertex belonging to subset T is identified as the external polyhedron of the building surface.

[0119] It should be noted that, in the above embodiments of this application, after constructing the directed graph, an ST cut is performed, that is, the vertex V in the directed graph is divided into two disjoint subsets S and T, such that the source vertex source∈S and the sink vertex sink∈T;

[0120] The st cut has low-order polynomial complexity. By minimizing the global energy, the minimum cut can be found efficiently (by calculating the minimum cut, the arithmetic problem is transformed into a mathematical problem of solving G = (V, E) which is a graph composed of vertices and edges). The graph cut (S,T) (the graph cut (S,T) is the st cut mentioned above) corresponds to the binary label of the second polyhedron - where S corresponds to the polyhedron inside the building surface, and T corresponds to the polyhedron outside.

[0121] Specifically, if Figure 15 As shown, the process of obtaining a subset S by minimizing global energy includes the following steps:

[0122] Step S441: Traverse each of the three-dimensional point cloud data: Obtain the visibility item E of each of the three-dimensional point cloud data. vis (S) and obtain the directional term E of each of the three-dimensional point cloud data. orient (S);

[0123] Step S442: Based on the visibility item E vis (S) and directional term E orient (S) Calculate and obtain the energy function E(S);

[0124] The energy function E(S) is calculated as follows:

[0125] E(S)=E vis (S)+E orient (S);

[0126] Step S443: Determine whether the energy function E(S) is greater than 0; if so, determine that the three-dimensional point cloud data is the three-dimensional point cloud data of the external polyhedron of the building surface; construct a subset S based on the three-dimensional point cloud data of all the external polyhedrons of the building surface (correspondingly, the vertices of the three-dimensional point cloud data of the external polyhedron of the building surface in the directed graph will be included in the subset S).

[0127] It should be noted that the above visibility item E vis (S) In the generation stage of dense point cloud (i.e., the above-mentioned 3D point cloud data), each 3D point cloud data can obtain image ray information, i.e., visibility information; the embodiments of this application utilize this visibility to construct a visual item E. vis (S), which increases the weight of the surface traversed by the line of sight (the weight is the aforementioned visibility term E). visThe value of (S); the principle is that if a point in the point cloud is on the surface of a building, then its corresponding line of sight, that is, the line formed by the point and the center of the photographic image, should be unobstructed; therefore, the weight of the edge can be assigned by the intersection of the line of sight and the polyhedron, as shown in the diagram. Figure 11 As shown.

[0128] like Figure 16 For each point p in the point cloud i Compare it with the photographic center c of the corresponding j-th visible image. ij Connecting these lines, we obtain the line of sight (represented by the red dashed line in the diagram); considering the scale accuracy σ, the line of sight starts from the photography center c. ij To point p i The direction extends the distance σ, resulting in a destination e. i S1 to S5 are the planes extracted in step S30; Cell1 to Cell9 are the second polyhedrons constructed in step S41.

[0129] From the photography center c ij The polyhedrons on the building surface should be visible from the starting line of sight, while those passing through the building surface should be invisible; therefore, polyhedra Cell2 and Cell4, which are on the same side as the direction of the normal vector of plane S5, are visible in the diagram, while polyhedron Cell7, which is on the opposite side of the direction of the normal vector of plane S5, is invisible; the area containing the photographic center c... ij The polyhedron Cell2 is connected to the source point with a weight of α. vis ; the endpoint e i The polyhedron Cell7, which contains the cell, is connected to the sink, and its weight is also α. vis Along the line of sight, all faces in the opposite direction will have their weight increased by α. vis Unless the point belongs to that surface;

[0130] The above direction term E orient (S), Since the edges of a directed graph have directions, it is necessary to consider the direction of the face corresponding to the edge;

[0131] The embodiment of this application uses the direction term E. orient (S) takes a zero value for surfaces whose normal vector direction is in the same direction as the line of sight (i.e., the angle between them is less than 90°), and takes an infinite value for surfaces whose normal vector direction is opposite to the line of sight (i.e., the angle between them is greater than 90°).

[0132] The dense point cloud (i.e., the aforementioned 3D point cloud data) generated for each building using low-altitude oblique imagery can reach millions of pixels. Without optimization of visibility calculation, it is difficult to apply this method to the reconstruction of lightweight building models at the city level. The visibility calculation process for each line of sight is independent of each other, which provides conditions for GPU parallelism. Therefore, in this embodiment, a GPU thread is allocated to each line of sight to perform visibility calculation.

[0133] In summary, the proposed method for automatically extracting building object bounding box models based on graph cuts in this invention first extracts planes from point clouds, utilizing the ray information of densely matched point clouds as prior information to transform building reconstruction into a graph cut problem, avoiding the huge computational burden caused by dynamic programming. These planes are then merged and regularized for optimization. Next, the bounding boxes of the point clouds are cut using planes to obtain candidate polyhedra. Finally, using visibility information, a graph cut model is constructed to binary-label the polyhedra, distinguishing between polyhedra inside and outside the building, thus achieving building bounding box model extraction. Experimental results verify that the algorithm of this application can achieve good results on noisy point clouds of buildings. Furthermore, this application achieves GPU parallelization of visibility weight calculation, thereby achieving the goal of efficiently generating building bounding box models, making the method applicable to batch building model reconstruction at the city level. It automatically generates lightweight single-building models, which are beneficial for storage, rendering, and subsequent GIS quantitative analysis, greatly reducing manual workload.

[0134] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; those skilled in the art can modify the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for automatically extracting building object frame models based on graph cutting, characterized in that: The steps are as follows: Acquire three-dimensional point cloud data of the current building; perform a plane extraction operation based on the three-dimensional point cloud data to obtain multiple planes corresponding to the building; Based on global regularization, all the planes are optimized and merged to obtain multiple new planes; constructing a polyhedron based on all of the new planes; Cutting the polyhedron to obtain multiple candidate planes; Constructing a plurality of second polyhedrons based on each of the candidate planes; Binary labeling is performed on the second polyhedron by visibility analysis to obtain an internal polyhedron on the building surface and an external polyhedron on the building surface; The building model is obtained by merging the internal polyhedrons of the building surface.

2. The method for automatically extracting building object frame models based on graph cuts according to claim 1, characterized in that: The step of optimizing and merging all the planes based on global regularization to obtain multiple new planes includes the following steps: Based on all the planes, multiple plane pairs (s i ,s j ); Calculate and obtain the plane pair (s i ,s j )’s included angle θ; Get each of the plane pairs (s i ,s j ) The number of support points N of each plane; Traverse each of the plane pairs (s i ,s j ), determine the plane pair (s i ,s j ) is less than or equal to the angle threshold θ t And the plane pair (s i ,s j ) is the number of support points N of each plane greater than or equal to the point threshold N t If so, then the plane pair (s i ,s j ) are merged to obtain a new plane; if not, the plane pair (s i ,s j ) are taken as new planes.

3. The method for automatically extracting building object frame models based on graph cuts according to claim 2 is characterized in that: The plane pair (s i ,s j ) represents two adjacent planes.

4. The method for automatically extracting building object frame models based on graph cuts according to claim 3 is characterized in that: The step of cutting the polyhedron to obtain a plurality of candidate planes comprises the following steps: Calculate and obtain the bounding box corresponding to all the current three-dimensional point cloud data; Extracting the coverage of the bounding box to obtain the coverage corresponding to each new plane; After arranging the respective coverage surfaces in descending order according to their area sizes, the respective coverage surfaces are traversed in order from large to small to determine whether the polyhedron intersects with the coverage surface. If so, the polyhedron is segmented using the coverage surface to obtain a plurality of candidate planes.

5. The method for automatically extracting building object frame models based on graph cuts according to claim 4 is characterized in that: The binary marking of the second polyhedron by visibility analysis to obtain an internal polyhedron on the building surface and an external polyhedron on the building surface includes the following steps: Constructing an empty initial directed graph; mapping the second polyhedral primitive corresponding to the second polyhedron to the initial directed graph to obtain a vertex V, and mapping the common faces of every two adjacent polyhedrons to the initial directed graph to obtain an edge E, thereby obtaining a directed graph; Add source points and sink points to each vertex V in the directed graph; obtain the weight w of each edge E; Based on the source point, sink point and the weight w, all the vertices V in the directed graph are processed by st-cut to obtain a subset S and a subset T; wherein the source point belongs to the subset S; the sink point belongs to the subset T; Based on the subset S, the internal polyhedron of the building surface is obtained, and based on the subset T, the external polyhedron of the building surface is obtained.

6. The method for automatically extracting building object frame models based on graph cuts according to claim 5 is characterized in that: After "adding a source point and a sink point to each vertex V in the directed graph respectively", it also includes: connecting the vertex to the source point and the sink point respectively.

7. The method for automatically extracting building object frame models based on graph cuts according to claim 6 is characterized in that: The subset S represents the vertices of the polyhedron inside the building surface; The subset T represents the vertices in the building surface exterior polyhedron.

8. The method for automatically extracting building object frame models based on graph cuts according to claim 7 is characterized in that: The step of processing all the vertices V in the directed graph by st-cut based on the source point, the sink point and the weight w to obtain a subset S and a subset T includes: first obtaining the subset S by distinguishing through global energy minimization, and then obtaining a subset T consisting of all vertices except the subset S.

9. The method for automatically extracting building object frame models based on graph cuts according to claim 8, characterized in that: The step of obtaining the subset S by distinguishing through global energy minimization includes the following steps: Traverse each of the three-dimensional point cloud data to obtain the visibility item E of each of the three-dimensional point cloud data vis (S) and obtain the directional item E of each of the three-dimensional point cloud data orient (S); Based on the visibility item E vis (S) and directional term E orient (S) Calculate and obtain the energy function E(S); Determine whether the energy function E(S) is greater than 0; if so, determine that the three-dimensional point cloud data is the three-dimensional point cloud data of the external polyhedron of the building surface; and construct a subset S based on the three-dimensional point cloud data of all the external polyhedrons of the building surface.

10. The method for automatically extracting building object frame models based on graph cuts according to claim 9, characterized in that: The energy function E(S) is calculated as follows: E(S)=E vis (S)+E orient (S)。

Citation Information

Patent Citations

  • Semantic perception triangulation network model building entity reconstruction method

    CN115861571A

  • Oblique photography three-dimensional model local flattening method based on filtering algorithm

    CN117372246A

  • Building multi-level-of-detail model reconstruction method based on multi-source data fusion

    CN118052938A

Cited By

  • House point cloud segmentation method and device

    CN120689615A