Building point cloud with constraint reconstruction method based on global alignment
By reconstructing building point clouds using a global alignment method, and employing initial normal estimation, sparse optimization, and region growing algorithms to construct geometric constraints, the problem of efficiency and constraint maintenance imbalance in existing technologies is solved, achieving efficient point cloud reconstruction.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- XIAMEN UNIV
- Filing Date
- 2023-08-18
- Publication Date
- 2026-07-07
AI Technical Summary
Existing technologies struggle to effectively balance constraint preservation and operational efficiency in building point cloud reconstruction, and their reliance on the reliability of initial primitive segmentation results is insufficient.
A global alignment-based approach is adopted, which optimizes the position and orientation of planar primitives by performing initial normal estimation, sparse optimization, region growing algorithm segmentation, and global optimization on the point cloud, combined with geometric constraints constructed by the principal axis direction.
It achieves improved reconstruction efficiency and robustness while maintaining strict geometric constraints, reduces dependence on initial primitive segmentation, and can better fit the input point cloud.
Smart Images
Figure CN117078852B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of point cloud 3D reconstruction, and in particular to a constrained reconstruction method for building point clouds based on global alignment, which takes into account the maintenance of geometric constraints. Background Technology
[0002] With the maturation of acquisition devices, point clouds are increasingly becoming an important part of geometric representation. 3D scanning equipment can acquire millions of point cloud data points; however, for most applications, this massive amount of data brings huge computational costs and increases the system's operational burden. Furthermore, since point clouds only contain information such as point coordinates, the topological relationships between data points are not clearly defined. In many practical applications, relying solely on this simple information is often insufficient to meet requirements; therefore, recovering the shape of the input object from the point cloud is essential. Point cloud reconstruction is a classic problem, and depending on its application scenario, there are many different algorithms, such as explicit reconstruction methods and implicit reconstruction methods. Within reconstruction problems, there is a special type of reconstruction that focuses primarily on 3D urban scenes: building reconstruction.
[0003] A key characteristic of building models is the presence of highly structured data, such as numerous planar structures. Therefore, a primary method for processing these models is geometric primitive-based reconstruction. First, shape detection is performed on the input point cloud to identify basic geometric primitives such as planes, cylinders, cones, and spheres. Based on these detected primitives, an algorithm is designed to ultimately output a combined model of these primitives. Another significant characteristic of building models is the presence of prior knowledge of constraints, such as parallel and perpendicular walls, and symmetrical structures designed for aesthetic or engineering requirements. This indicates that when reconstructing man-made objects, we need to consider not only the accuracy of the fit but also these prior constraints—that is, constrained point cloud reconstruction.
[0004] The general approach to constrained point cloud reconstruction is to first detect and then optimize. That is, firstly, basic primitives are extracted from the input point cloud and the constraint relationships between the primitives are detected. Secondly, the constraint relationships obtained in the previous step are used to adjust and optimize the parameters of each primitive to obtain the final result. Summary of the Invention
[0005] The purpose of this invention is to provide a building point cloud reconstruction method that takes into account geometric constraints, which can maintain geometric constraint relationships during the reconstruction process and obtain better reconstruction results.
[0006] This invention includes the following steps:
[0007] 1) Estimate the initial normal of the input point cloud;
[0008] 2) Perform sparsity optimization on the estimated initial normals so that points on the same face have the same normal;
[0009] 3) Combining the initial estimated normal and the optimized normal, the input point cloud is segmented using a region growing algorithm to obtain a series of planar primitives;
[0010] 4) Calculate the principal axis direction of the building point cloud as a global reference, and set the target normal for each planar primitive to construct the parallel, perpendicular, coplanar and symmetric relationships in the planar primitive;
[0011] 5) Based on the constructed constraints, globally optimize the orientation and position of the planar primitives;
[0012] 6) Find the intersection of all planes to calculate the boundary of the plane primitives and obtain the final model.
[0013] In step 1), input the artificial point cloud P = {p i For each point in the point cloud, find the k nearest points to it as its local neighborhood, and use PCA (Principal Component Analysis) to calculate the plane normal fitted to the neighborhood points as the initial normal n of the point. i .
[0014] In step 2), the goal of normal optimization is to optimize the normal vector at each point. and initial normal n i The points should be sufficiently close, with their neighborhood points having the same normal vectors as much as possible. Furthermore, to compute the local coordinate system of the point cloud, the number of optimized normal vectors in different directions is limited to 3. The objective function for optimization is shown below:
[0015]
[0016] in It is point p i k nearest neighbor, It is a set composed of normals in different directions after optimization.
[0017] In step 2), the optimization process of the objective function is divided into two steps. First, without considering the constraints, a general normal sparse optimization problem is solved. Then, a subset of the optimized normal set is selected to satisfy the constraints.
[0018] In step 3), an improved region growing algorithm is used to segment the input point cloud into primitives. Since man-made objects mainly consist of planes, only the segmentation of planar primitives is considered during the segmentation process. During region growing, initial points are randomly selected as the growing region, and the region is grown based on the approximation of the normal vectors of the points and their neighbors. Then, at the end of the growing process, planar primitives are fitted to the region points, and the growing process is repeated until the number of points is less than a certain specific value.
[0019] In step 3), when measuring the approximation of the normal during the growth process, both the initial normal and the optimized normal are considered. When the optimized normal of the selected point is the same as that of the points in the region and the initial normal is close, the selected point is added to the growth region.
[0020] In step 4), the input point cloud is divided into 3 groups P = P1∪P2∪P3 according to the optimized normal, and the error between each group and the initial normal is calculated as follows:
[0021]
[0022] In step 4), based on the calculated error e(P) i Let the magnitude of the error be defined, and let v, w, w be the principal axis directions of the point cloud. Here, u is the optimized normal corresponding to the group with the smallest error, and w is the optimized normal corresponding to the group with the largest error. Then, the principal axis directions of the point cloud are orthogonalized (u, w, w), i.e., u is kept constant while v and w are corrected, resulting in three mutually orthogonal principal axis directions.
[0023] In step 4), the planar primitive f i normal direction The target normal is set to the corresponding orthogonal principal axis direction and matched with the principal axis directions u, v, w. This constrains the parallelism between all planes matched with the same principal axis direction and the perpendicularity between all planes matched with different principal axis directions. The target normal is set as follows:
[0024]
[0025] In step 4), a coplanar relationship is constructed by adding a distance threshold judgment to the parallel planar primitives, such as for two planes f matched to u. i ,f j If the distance between them is less than a certain threshold, the two planes are merged to obtain a new plane.
[0026] In step 4), the normal of the planar primitive is transformed to... In the coordinate system, and respectively search for whether there exists any coordinate plane. The symmetry relationship, for the normals of two planar primitives They transform to The normal vector in the coordinate system is The corresponding coordinates are If they satisfy the symmetry condition, then update them to average normals while maintaining strict symmetry, and inversely transform the updated normals back to the original coordinate system, setting them as the target normals of the planar primitives. The symmetry condition is as follows:
[0027]
[0028] The updated normal vector is shown below:
[0029]
[0030] The target normal of the planar primitive is set as follows:
[0031]
[0032] in
[0033] In step 5), the position and orientation of the planar primitives are optimized based on the constructed geometric constraints to obtain the final result. During the optimization process, the normal of each planar primitive is first set as its target normal. Then, all planar primitives are rotated as a whole to maintain the geometric constraints, and each planar primitive is translated individually to ensure a good fit to the input points. The optimization objective is to minimize the sum of the distances from the point cloud to the planar primitives. The objective function is shown below:
[0034]
[0035] in:
[0036]
[0037] Where α, β, θ are rotation angles, and R(α), R(β), R(θ) are rotation matrices.
[0038] In step 6), firstly, all planes are intersected pairwise to obtain a set of candidate planes. Then, a closed model is selected from the set of candidate planes as the final result.
[0039] Compared with the prior art, the present invention has the following outstanding advantages:
[0040] 1. Existing constrained reconstruction techniques cannot effectively balance constraint preservation and operational efficiency. Techniques with good constraint preservation have long running times and depend on the initial primitive segmentation results; while fast techniques cannot maintain strict geometric constraints. This invention can achieve good operational efficiency while ensuring strict geometric constraints.
[0041] 2. The present invention designs an improved region growing algorithm for point cloud primitive segmentation, taking into account both the initial normal and the optimized normal, and has a certain robustness to noise and outliers.
[0042] 3. This invention proposes a method for calculating the principal axis direction of building point clouds and uses it as a global reference direction to uniformly constrain the relationship between all planar primitives, thus avoiding the contradictory constraint problems that may arise from pairwise comparisons. Attached Figure Description
[0043] Figure 1 This describes a method for reconstructing building point clouds with constraints based on global alignment.
[0044] Figure 2 The input is a point cloud model of the building.
[0045] Figure 3 Visualize the optimized normal and principal axis directions.
[0046] Figure 4 This represents the segmentation result of planar primitives.
[0047] Figure 5 This is an optimized schematic diagram for rotation and translation.
[0048] Figure 6 A schematic diagram of the process of finding intersections in a plane. Detailed Implementation
[0049] To make the objectives, technical solutions, and advantages of the present invention clearer, the following embodiments will be used to further illustrate the present invention in conjunction with the accompanying drawings.
[0050] Before describing this embodiment in detail, it should be noted that the building point cloud with constraint reconstruction method based on global alignment demonstrated in this embodiment is an automatic method, in which the user only needs to provide a building point cloud model as input.
[0051] like Figure 1 This invention provides a constrained reconstruction method for building point clouds based on global alignment. The method involves inputting a building point cloud, performing planar primitive segmentation using an improved region growing algorithm, where each color represents a plane. Geometric constraints are then constructed: planes of the same color are parallel, and planes of different colors are perpendicular. Finally, the result is optimized based on these constraints. This embodiment specifically includes the following steps:
[0052] S1. Estimate the initial normal of the input point cloud;
[0053] The input building point cloud is as follows Figure 2 As shown, the input is an artificial point cloud P = {p i For each point in the point cloud, find the k nearest points to it as its local neighborhood, and use PCA (Principal Component Analysis) to calculate the plane normal fitted to the neighborhood points as the initial normal n of the point. i .
[0054] S2. Perform sparse optimization on the estimated initial normals so that points on the same face have the same normal.
[0055] like Figure 3 As shown, the goal of normal optimization is to optimize the normal vector at each point. and initial normal n i The points should be sufficiently close, with their neighborhood points having the same normal vectors as much as possible. Furthermore, to compute the local coordinate system of the point cloud, the number of optimized normal vectors in different directions is limited to 3. The objective function for optimization is shown below:
[0056]
[0057] in It is point p i k nearest neighbor, It is a set composed of normals in different directions after optimization.
[0058] The optimization process of the objective function is divided into two steps. First, without considering the constraints, we solve a general normal sparse optimization problem. Then, we select a subset of the optimized normal set that satisfies the constraints.
[0059] S3. Combining the initial estimated normal and the optimized normal, the input point cloud is segmented using a region growing algorithm to obtain a series of planar primitives.
[0060] like Figure 4 As shown, an improved region growing algorithm is used to segment the input point cloud into primitives. Since man-made objects mainly consist of planes, only the segmentation of planar primitives is considered during the segmentation process. During region growing, initial points are randomly selected as the growing region, and the region is grown based on the approximation of the normal vectors of the points and their neighbors. Then, at the end of the growing process, planar primitives are fitted to the region points, and the growing process is repeated until the number of points is less than a certain specific value.
[0061] When measuring the approximation of the normal during the growth process, both the initial normal and the optimized normal are considered. When the optimized normal of the selected point is the same as that of the points in the region and the initial normal is close, the selected point is added to the growth region.
[0062] S4. Calculate the principal axis direction of the building point cloud as a global reference, and set the target normal for each planar primitive to construct the parallel, perpendicular, coplanar and symmetric relationships in the planar primitive.
[0063] The input point cloud is divided into 3 groups P = P1∪P2∪P3 based on the optimized normal, and the error between each group and the initial normal is calculated as follows:
[0064]
[0065] Based on the calculation error e(P) iLet the magnitude of the error be used to define the principal axis directions u, v, and w of the point cloud, where u is the optimized normal corresponding to the group with the smallest error, and w is the optimized normal corresponding to the group with the largest error. Then, the principal axis directions of the point cloud are orthogonalized by u, v, and w, i.e., keeping u unchanged and correcting v and w respectively, finally obtaining three mutually orthogonal principal axis directions.
[0066] Planar primitive f i normal direction The target normal is set to the corresponding orthogonal principal axis direction and matched with the principal axis directions u, v, w. This constrains the parallelism between all planes matched with the same principal axis direction and the perpendicularity between all planes matched with different principal axis directions. The target normal is set as follows:
[0067]
[0068] Coplanar relationships are constructed by adding a distance threshold to parallel planar primitives, such as for two planes f matched to u. i ,f j If the distance between them is less than a certain threshold, the two planes are merged to obtain a new plane.
[0069] Transform the normal of the planar primitive to In the coordinate system, and respectively search for whether there exists any coordinate plane. The symmetry relationship, for the normals of two planar primitives They transform to The normal vector in the coordinate system is The corresponding coordinates are If they satisfy the symmetry condition, then update them to average normals while maintaining strict symmetry, and inversely transform the updated normals back to the original coordinate system, setting them as the target normals of the planar primitives. The symmetry condition is as follows:
[0070]
[0071] The updated normal vector is shown below:
[0072]
[0073] The target normal of the planar primitive is set as follows:
[0074]
[0075] in,
[0076] S5. Based on the constructed constraint relationships, globally optimize the orientation and position of the planar primitives.
[0077] The final result is obtained by optimizing the position and orientation of the planar primitives based on the constructed geometric constraints. For example... Figure 5 As shown, in Figure 5 In the diagram, from left to right: setting the target normal as the initial direction; overall rotation during optimization; the rotated model; and individual translations during optimization (see the plane on the right). During optimization, first, the normal of each planar primitive is set as its target normal. Then, all planar primitives are rotated to maintain geometric constraints, and each planar primitive is translated individually to ensure proper fitting of the input points. The optimization objective is to minimize the sum of distances from the point cloud to the planar primitives. The objective function is shown below:
[0078]
[0079] in:
[0080]
[0081] Where α, β, θ are rotation angles, and R(α), R(β), R(θ) are rotation matrices.
[0082] S6. Find the intersection of all planes and calculate the boundary of the plane primitives to obtain the final model.
[0083] like Figure 6 As shown, firstly, the intersection of all planes is calculated pairwise to obtain a set of candidate planes. Next, a closed model is selected from the candidate plane set as the final result. Figure 6 In the middle, from left to right, are: the optimized planar primitives; the candidate set obtained by intersecting the planes; and the finally selected closed model (which determines the boundary of the plane).
[0084] This invention proposes a constrained reconstruction method for building point clouds based on global alignment. This method performs sparsity optimization on the normals of the point cloud and utilizes an improved region growing algorithm to segment planar primitives. By calculating the principal axis directions of the building point cloud, the geometric constraints existing in the model are globally constructed, and a target normal is set for each planar primitive. Finally, a nonlinear optimization method is used to obtain the final result that conforms to the geometric constraints. The reconstruction method of this invention can fit the input points as closely as possible while maintaining strict geometric constraints, achieving good results.
Claims
1. A method for constrained reconstruction of building point clouds based on global alignment, characterized in that, Includes the following steps: 1) Estimate the initial normal of the input point cloud; 2) Perform sparsity optimization on the estimated initial normals so that points on the same face have the same normal; 3) Combining the initial estimated normal and the optimized normal, the input point cloud is segmented using a region growing algorithm to obtain a series of planar primitives; 4) Calculate the principal axis direction of the building point cloud as a global reference, and set the target normal for each planar primitive to construct the parallel, perpendicular, coplanar, and symmetric relationships in the planar primitive. The specific steps are as follows: Based on the optimized normal, the input point cloud is divided into 3 groups. And calculate the error between each group and the initial normal, as shown below: Based on calculation error The size of each point cloud is used to define its principal axis direction. ,in It is the optimized normal corresponding to the group with the smallest error. The optimized normal vector corresponding to the group with the largest error; Perform on the principal axis direction of the point cloud Orthogonalization, i.e., preserving No change, but be modified separately. This ultimately yields three mutually orthogonal principal axis directions. ; planar primitives normal direction With the main axis direction Perform matching and set the corresponding target normal to its corresponding orthogonal principal axis direction. In this way, constrain the parallel relationship between all planes matched to the same principal axis direction, and constrain the perpendicular relationship between all planes matched to different principal axis directions. The target normal is set as follows: Coplanar relationships are constructed by adding a distance threshold to parallel planar primitives, such as for matched primitives. Two planes If the distance between them is less than a certain threshold, the two planes are merged to obtain a new plane; Transform the normal of the planar primitive to In the coordinate system, search for whether there exists any coordinate plane. The symmetry relationship, for the normals of two planar primitives They transformed to The normal vector in the coordinate system is The corresponding coordinates are If they satisfy the symmetry condition, then update them to average normals while maintaining strict symmetry, inversely transform the updated normals back to the original coordinate system, and set them as the target normals of the planar primitives; where the symmetry condition is as follows: The updated normal vector is shown below: The target normal of the planar primitive is set as follows: in, ; 5) Based on the constructed constraints, globally optimize the orientation and position of the planar primitives; The global optimization of the orientation and position of the planar primitives involves first setting the normal of each planar primitive as its target normal, then rotating all planar primitives to maintain geometric constraints, and translating each planar primitive individually to ensure fitting of the input points. The optimization objective is to minimize the sum of the distances from the point cloud to the planar primitives. The objective function is as follows: in: in, The rotation angle is... It is a rotation matrix; 6) Find the intersection of all planes to calculate the boundary of the plane primitives and obtain the final model.
2. The building point cloud reconstruction method with constraints based on global alignment as described in claim 1, characterized in that, In step 1), the initial normal estimation of the input point cloud specifically involves: inputting an artificial point cloud. For each point in the point cloud, find the nearest neighbor. Each point is taken as a local neighborhood, and the plane normal fitted by the neighborhood points is calculated using principal component analysis, which is then used as the initial normal of the points. .
3. The building point cloud reconstruction method with constraints based on global alignment as described in claim 1, characterized in that, In step 2), the sparse optimization of the estimated initial normal is to make the optimized normal of each point... and initial normal The points should be sufficiently close, with their neighborhood points having the same normal vectors as much as possible. Furthermore, to compute the local coordinate system of the point cloud, the number of optimized normal vectors in different directions is limited to 3. The objective function for optimization is shown below: in, It is a point of Neighbor, It is a set composed of normals in different directions after optimization; The optimization process of the objective function consists of two steps. First, without considering any constraints, we solve a general normal sparse optimization problem. Second, we select a subset of the optimized normal set that satisfies the constraints.
4. The building point cloud reconstruction method with constraints based on global alignment as described in claim 1, characterized in that, In step 3), the input point cloud is segmented using the region growing algorithm to obtain a series of planar primitives. Specifically, the input point cloud is segmented using an improved region growing algorithm, and since man-made objects mainly contain planes, only the segmentation of planar primitives is considered during the segmentation process. During the region growth process, an initial point is randomly selected as the growth region, and the region is grown according to the approximation of the normal of the point and its neighboring points. Then, at the end of the growth, a plane primitive is fitted to the region points, and the growth process is repeated until the number of points is less than a certain specific value. When measuring the approximation of the normal during the growth process, both the initial normal and the optimized normal are considered. When the optimized normal of the selected point is the same as that of the points in the region and the initial normal is close, the selected point is added to the growth region.
5. The building point cloud reconstruction method with constraints based on global alignment as described in claim 1, characterized in that, In step 6), the boundary of the plane primitive is calculated by finding the intersection of all planes. First, the candidate plane set is obtained by finding the intersection of all planes pairwise. Then, a closed model is selected from the candidate plane set as the final result.