Sparse geometric feature-oriented house point cloud interactive registration method

Abstract

The application discloses a house point cloud interactive registration method for sparse geometric features, and belongs to the field of point cloud registration. The method firstly increases point features by uniformly pasting reflective points, and then judges the overlapping degree of two adjacent point clouds. If the overlapping degree is less than a first preset threshold, corresponding points are selected in the overlapping part of the target point cloud and the source point cloud, and then coarse registration is performed. If the overlapping degree is greater than or equal to the first preset threshold and less than or equal to a second preset threshold, reflective points are selected in the center neighborhood of the target point cloud and the source point cloud, and then coarse registration is performed. If the overlapping degree is greater than the second preset threshold, global registration is used to perform coarse registration on the target point cloud and the source point cloud. After coarse registration, ICP fine registration is performed. According to the different point cloud overlapping degrees, different coarse registration technologies are adopted, the automatic registration technology and the manual interactive registration technology are combined, accurate initial information is provided for fine registration, and the registration result precision is improved.

Description

Technical Field

[0001] This invention relates to the field of point cloud registration, and in particular to an interactive registration method for house point clouds oriented towards sparse geometric features. Background Technology

[0002] During the smart home design phase, when the house is still in its unfinished state, single-frame point clouds of the house are obtained from multiple perspectives through point cloud acquisition, specifically including walls, doors, windows, water and electricity meters, pipes, etc. Then, through point cloud processing and other methods, the single-frame point clouds are registered and reconstructed into complete interior point clouds, constructing a 3D model of the house interior.

[0003] Single-frame house point clouds are sparse. Existing registration techniques for sparse feature point clouds employ a single point cloud registration method. This involves coarse registration of two adjacent point clouds to roughly align them, providing initial orientation and position. Fine registration is then performed using the iterative nearest neighbor (ICP) algorithm based on the initial values ​​from the coarse registration. However, if the acquired object has a large planar area (such as house walls), relying solely on a single coarse registration technique results in low accuracy, potentially leading to incorrect registration direction or misalignment. This provides incorrect initial information for subsequent fine registration, resulting in prolonged registration time and even registration failure.

[0004] Currently, mainstream point cloud coarse registration algorithms are mainly divided into two categories: registration methods based on global search and registration methods based on geometric feature description. Geometric feature-based registration methods first extract features from the image information and then use these features as a model for registration. Local features have interrelationships, such as geometric relationships, radiometric relationships, and topological relationships. These relationships can be used to describe global features. Local feature-based registration is usually based on points, lines, or edges, while global feature-based registration utilizes the relationships between local features. Global search-based registration methods typically randomly select several points from the original data and find corresponding points in the target data through an exhaustive search. All possible transformation matrices are calculated, and the optimal transformation is determined by voting or selecting the method with the minimum error function. This approach, considering all possible correspondences, can achieve good registration results but often incurs a large computational load. The most commonly used algorithm framework is RANSAC (Random Sample Consensus). Summary of the Invention

[0005] The purpose of this invention is to provide an interactive registration method for house point clouds based on sparse geometric features. This method can select an appropriate point cloud registration method based on the geometric features of the collected point cloud, thereby improving the accuracy of the registration results.

[0006] To achieve the above objectives, the present invention provides the following solution:

[0007] An interactive registration method for house point clouds based on sparse geometric features includes:

[0008] Evenly paste reflective dots on the plane of the object being scanned inside the building, and collect multi-frame point clouds of the object being scanned;

[0009] Select one frame of point cloud as the target point cloud, and the next frame of point cloud adjacent to the target point cloud as the source point cloud;

[0010] Determine the overlap between the target point cloud and the source point cloud;

[0011] If the overlap is less than the first preset threshold, then the corresponding point is selected in the overlapping part of the target point cloud and the source point cloud, and the target point cloud and the source point cloud are coarsely registered according to the selected corresponding point, and then ICP fine registration is performed.

[0012] If the overlap is greater than or equal to the first preset threshold and less than or equal to the second preset threshold, then reflective points are selected in the centered neighborhood of the target point cloud and the source point cloud, and coarse registration is performed on the target point cloud and the source point cloud based on the selected reflective points, followed by ICP fine registration; wherein, the second preset threshold is greater than the first preset threshold.

[0013] If the overlap is greater than the second preset threshold, then global registration is used to perform coarse registration between the target point cloud and the source point cloud, and then ICP fine registration is performed.

[0014] Select the registered source point cloud as the new target point cloud, and return to the step "the point cloud adjacent to the target point cloud as the source point cloud", until all frame point clouds are traversed and all frame point clouds are aligned.

[0015] According to specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0016] This invention discloses an interactive registration method for house point clouds with sparse geometric features. First, to address the problem of sparse geometric features in point clouds, reflective points are uniformly pasted to increase point features. Then, the overlap between two adjacent point cloud frames is determined. Based on the different overlaps, different coarse registration techniques are adopted. By combining automatic registration techniques and manual interactive registration techniques, accurate initial information is provided for fine registration, thus improving the accuracy of the registration results. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 The flowchart illustrates an interactive registration method for house point clouds based on sparse geometric features, as provided in an embodiment of the present invention. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] This invention proposes an interactive registration method for house point clouds based on sparse geometric features, which solves the problem of low overall accuracy or even registration errors caused by inaccurate coarse registration in point cloud registration with sparse geometric features.

[0021] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0022] like Figure 1 As shown in the figure, an interactive registration method for house point clouds oriented towards sparse geometric features provided by an embodiment of the present invention includes:

[0023] Step 1: Evenly paste reflective dots on the flat surface of the object to be scanned inside the house, and collect multi-frame point clouds of the object to be scanned.

[0024] This invention is primarily applied in the smart home design phase. When the house is still in its unfinished state, single-frame point clouds of the house are acquired from multiple perspectives through point cloud acquisition, specifically including walls, doors, windows, water and electricity meters, pipes, etc. Then, using point cloud processing methods, the single-frame point clouds are registered and reconstructed into a complete interior point cloud, constructing a 3D model of the house's interior. Before acquiring the point cloud, reflective dots are pasted onto the plane of the object being scanned. The density of the reflective dots is uniform, and during structured light acquisition, it is necessary to ensure that two adjacent point clouds contain three or more corresponding points.

[0025] Structured light: By using pre-designed patterns with special structures (such as discrete light spots, striped light, coded structured light, etc.), the patterns are projected onto the surface of a 3D object. A separate camera observes the distortion of the image on the 3D physical surface. If the object's surface is not planar, the observed structured light pattern will exhibit different distortions due to the object's varying geometry, and these distortions will differ depending on the distance. Based on the known structured light pattern and the observed distortions, algorithms can be used to calculate the 3D shape and depth information of the object being measured.

[0026] After acquiring the point cloud, preprocessing is required. Preprocessing mainly consists of two parts: point cloud downsampling and denoising. Because structured light acquisition results in a large number of point clouds, subsequent point cloud denoising and configuration become too slow. Random downsampling is used to reduce the number of point clouds. Point cloud filtering primarily involves automatic filtering methods such as statistical filtering and Gaussian filtering for denoising. Since the acquired point cloud boundaries may be tilted or curved, and there may be noise that deviates and is difficult to remove using filtering methods, interactive point and line selection is combined to filter out excessively deviated noise and tilted or curved point cloud boundaries.

[0027] The preprocessing process is roughly as follows: downsample the acquired point cloud from multiple frames; use an automatic filtering method to denoise the downsampled point cloud; and use interactive point and line selection methods to filter out tilted and curved point cloud boundaries.

[0028] Step 2: Select a point cloud frame as the target point cloud and a point cloud frame adjacent to the target point cloud as the source point cloud.

[0029] Step 3: Determine the overlap between the target point cloud and the source point cloud.

[0030] Because the reflective points are roughly evenly distributed across the acquired plane, the degree of overlap can be determined by the overlap ratio of the corresponding reflective points. Since the acquisition range remains constant during the acquisition process and the reflective points are evenly distributed, the number of reflective points acquired in each frame of the point cloud is approximately the same. The overlap between two frame point clouds is as follows:

[0031]

[0032] In the formula, S represents the degree of overlap, m represents the number of overlapping reflective points between the two point clouds, and M represents the sum of the number of reflective points between the two point clouds.

[0033] Determine the overlap between two point clouds and use different coarse registration methods depending on the degree of overlap.

[0034] Step 4: If the overlap is less than the first preset threshold, select the corresponding point in the overlapping part of the target point cloud and the source point cloud, and perform coarse registration of the target point cloud and the source point cloud based on the selected corresponding point, and then perform ICP fine registration.

[0035] In this example, we will use a first preset threshold of 20% to illustrate the registration process in step 4.

[0036] When the overlap of point clouds is less than 20%, manual point selection coarse registration is adopted. By selecting the reflection points of corresponding points in two adjacent point clouds and the corner points in the point clouds, the rotation matrix R and translation matrix T of the point clouds are calculated so that the two point clouds are roughly aligned.

[0037] The transformation between point sets and point set coordinate systems is achieved by establishing a system of transformation equations through the relationship between each point in a point set (target point cloud) and the corresponding point in another point set (source point cloud).

[0038] The coordinate relationship of corresponding points in the two point cloud datasets can be described as follows:

[0039] Q = R 3×3 P+T 3×1 ;

[0040] Among them, R 3×3 T represents the rotation matrix at the corresponding point. 3×1 Let P represent the translation matrix of the corresponding point, and let Q represent the point cloud that has been transferred to the target coordinates after being rotated and translated by the rotation and translation matrices.

[0041] Corresponding points: There is an overlap between the two sets of point clouds. Point a in point cloud A corresponds to X in the real world, and point b in point cloud B also corresponds to X in the real world. Therefore, point a in point cloud A and point b in point cloud B are corresponding points.

[0042] The point selection method is implemented by designing interactive point cloud software using QT and C++. The interface displays two imported point clouds with overlapping parts during shooting. Points are selected from the overlapping parts, and the center of the reflective point of the selected point is the center of the overlapping part.

[0043] ICP (Inter-Centered Point Registration) precise registration: Theoretically, at least three pairs of non-collinear corresponding points are needed to solve for the rotation and translation matrices. However, for more accurate results, more corresponding point pairs are better. But in general, it's difficult to find accurate corresponding point pairs between two point cloud datasets. Therefore, an objective function is added to optimize the estimation of the rotation and translation matrices. Currently, the most commonly used objective function is the sum of squared Euclidean distances between corresponding points. For the selected point p... i Applying an initial transformation R, we obtain p' i Find the distance point p' i The nearest point q iThis forms corresponding point pairs. If E(R,T) does not satisfy the set minimum value, the iteration continues. When E(R,T) satisfies the minimum value, the obtained R and T are considered to be the optimal rotation matrix R and translation matrix T.

[0044] Based on this, the specific steps for ICP fine registration are as follows:

[0045] Construct the objective function as follows In the formula, E(R,T) represents the sum of squares of the Euclidean distances between corresponding points, R represents the rotation matrix, T represents the translation matrix, and n represents the number of points selected.

[0046] Select a point p in the overlapping region of the source point cloud. i ; where p i The selected point is the i-th point;

[0047] For a selected point p i Applying the rotation matrix R, we obtain p i Projection point p' in the target point cloud i ;

[0048] Find the distance projection point p' in the target point cloud. i The nearest point q i As p i The corresponding points form corresponding point pairs;

[0049] Based on all corresponding point pairs, with the goal of minimizing the sum of squared Euclidean distances between corresponding points, the objective function is solved iteratively using the least squares method to obtain the optimal rotation and translation matrices.

[0050] The source and target point clouds are precisely registered using the optimal rotation and translation matrices to align the point clouds in adjacent frames.

[0051] Step 5: If the overlap is greater than or equal to the first preset threshold and less than or equal to the second preset threshold, then select a reflective point in the neighborhood of the center of the target point cloud and the source point cloud, and perform coarse registration of the target point cloud and the source point cloud based on the selected reflective point, and then perform ICP fine registration; wherein, the second preset threshold is greater than the first preset threshold.

[0052] In this example, the registration process in step 5 is illustrated by taking a first preset threshold of 20% and a second preset threshold of 40% as an example.

[0053] Coarse registration: When the point cloud overlap is between 20% and 40%, the center of the reflected point is extracted manually. The reflected point is selected within the neighborhood of the center of the target point cloud and the source point cloud, specifically including:

[0054] Determine the centers of the target point cloud and the source point cloud respectively;

[0055] Using the center of the target point cloud or the center of the source point cloud as seed points, a region growing algorithm is used to determine the center neighborhood of the target point cloud and the center neighborhood of the source point cloud, respectively, based on curvature and color criteria.

[0056] Select multiple corresponding reflective points in the center neighborhood of the target point cloud and the center neighborhood of the source point cloud.

[0057] Then, by registering the selected points, the rotation matrix R and translation matrix T of the point cloud are calculated. The original point cloud is then transformed by the rotation matrix R and translation matrix T so that the two point clouds are roughly aligned.

[0058] Fine registration: The fine registration method in step 5 is the same as that in step 4, and will not be repeated here.

[0059] Step 6: If the overlap is greater than the second preset threshold, then perform coarse registration of the target point cloud and the source point cloud using global registration, and then perform fine registration using ICP.

[0060] In this example, we will use a second preset threshold of 40% to illustrate the registration process in step 6.

[0061] When the overlap of point clouds is higher than 40%, the two adjacent point clouds contain more features. Global registration can be used to perform coarse registration, calculate the rotation matrix R and translation matrix T of the point clouds, and make the two point clouds roughly aligned.

[0062] After obtaining roughly aligned point clouds through coarse registration, fine registration using ICP is performed. ICP is widely used in fine point cloud registration due to its advantages of computational simplicity, intuitiveness, and high registration accuracy. However, the algorithm's running speed and convergence to global optimization largely depend on the initial transformation estimate given in the coarse registration and the establishment of the correspondence during the iteration process. Using different coarse registration methods based on the different overlap levels of the point clouds can provide better positioning and avoid the iteration getting trapped in local optima.

[0063] ICP determines the corresponding point set by searching for the nearest point in the target point cloud from the source point cloud, and then iteratively solves for the optimal rotation matrix R and translation matrix T using the least squares method.

[0064] Step 7: Select the registered source point cloud as the new target point cloud, and return to step "the point cloud adjacent to the target point cloud as the source point cloud", until all frame point clouds are traversed and all frame point clouds are aligned.

[0065] For example, point cloud 1, point cloud 2, point cloud 3, and point cloud 4 were collected. Point cloud 1 is the target point cloud, and point cloud 2 is the source point cloud. Point cloud 1 and point cloud 2 are registered, which means transforming point cloud 2 to the coordinates of point cloud 1. Then point cloud 2 and point cloud 3 are registered, and finally point cloud 3 and point cloud 4 are registered.

[0066] The advantages of this invention are as follows:

[0067] 1. To address the issue of sparse geometric features in point clouds, reflective points are manually pasted to increase point features;

[0068] 2. To address the issues of excessive point cloud noise and uneven point density distribution, an automatic statistical Gaussian filtering method and an interactive point and line selection method were adopted. This noise filtering method can filter out most of the point cloud noise, preventing noisy points from being used as erroneous points in point cloud registration, thus ensuring the smooth registration of subsequent point clouds.

[0069] 3. Existing point cloud registration techniques employ a single coarse registration method to roughly align two point clouds before performing ICP fine registration. Relying solely on a coarse registration technique results in low accuracy and may even lead to incorrect registration direction or misalignment, providing incorrect initial information for subsequent fine registration, resulting in lengthy registration times and potential failures. Automatic registration is fast but requires high overlap between point clouds; interactive registration has lower overlap requirements but requires manual operation and is time-consuming. This invention employs different coarse registration methods based on varying point cloud overlap, combining automatic registration and manual interactive registration techniques to provide accurate initial information for fine registration.

[0070] Definitions:

[0071] Sparsity: Ideally, the feature values ​​of points on the same or similar surfaces will be very similar. Point clouds collected from the walls and floors of houses lack features, i.e., feature sparsity.

[0072] Point cloud features: Point cloud features are commonly represented by point cloud descriptors. Based on spatial scale, point cloud features can be categorized into: single-point features, local features, and global features. Single-point features include: 3D coordinates, normals, principal curvatures, eigenvalues, etc.; local features include: PFH, FPFH, SHOT, intensity gradient, etc.; global features include: VFH, CVFH, elevation difference, average intensity, etc.

[0073] Point cloud registration: Input two point clouds Ps (source) and Pt (target), output a transformation T (rotation R and translation t) that maximizes the overlap between Ps and Pt. It mainly consists of two steps: coarse registration and fine registration. Coarse registration refers to a relatively rough registration when the transformation between the two point clouds is completely unknown, mainly to provide a better initial transformation value for fine registration. Fine registration, given an initial transformation, further optimizes to obtain a more accurate transformation.

[0074] Interactive processing: The program pauses at a certain stage, awaiting user intervention to change the program's execution flow and state until the user's expected result is achieved. In point cloud registration, points are selected manually and interactively, and the relative positions of multiple corresponding points are calculated to obtain the rotation matrix R and the translation matrix t.

[0075] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0076] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. An interactive registration method for house point clouds oriented towards sparse geometric features, characterized in that, include: Evenly paste reflective dots on the plane of the object being scanned inside the building, and collect multi-frame point clouds of the object being scanned; Select one frame of point cloud as the target point cloud, and the next frame of point cloud adjacent to the target point cloud as the source point cloud; Determine the overlap between the target point cloud and the source point cloud; If the overlap is less than the first preset threshold, then the corresponding point is selected in the overlapping part of the target point cloud and the source point cloud, and the target point cloud and the source point cloud are coarsely registered according to the selected corresponding point, and then ICP fine registration is performed. If the overlap is greater than or equal to the first preset threshold and less than or equal to the second preset threshold, then reflective points are selected in the centered neighborhood of the target point cloud and the source point cloud, and coarse registration is performed on the target point cloud and the source point cloud based on the selected reflective points, followed by ICP fine registration; wherein, the second preset threshold is greater than the first preset threshold. If the overlap is greater than the second preset threshold, then global registration is used to perform coarse registration between the target point cloud and the source point cloud, and then ICP fine registration is performed. Select the registered source point cloud as the new target point cloud, and return to the step "the point cloud adjacent to the target point cloud as the source point cloud", until all frame point clouds are traversed and all frame point clouds are aligned. The first preset threshold is 20%, and the second preset threshold is 40%. If the overlap is less than a first preset threshold, then corresponding points are selected in the overlapping portion of the target point cloud and the source point cloud, and coarse registration is performed between the target point cloud and the source point cloud based on the selected corresponding points, including: When the overlap of point clouds is less than 20%, manual point selection coarse registration is adopted. By selecting the reflection points of corresponding points in two adjacent point clouds and the corner points in the point clouds, the rotation matrix R and translation matrix T of the point clouds are calculated to align the two point clouds. The transformation between point sets and point set coordinate systems is achieved by establishing a system of transformation equations through the relationship between each point in one point set and the corresponding point in another point set; The coordinate relationship of corresponding points in the two point cloud datasets is described as follows: Q=R 3×3 P+T 3×1 ; Among them, R 3×3 T represents the rotation matrix at the corresponding point. 3×1 Let P represent the translation matrix of the corresponding point, and let Q represent the point cloud that has been transferred to the target coordinates after being rotated and translated by the rotation and translation matrices. If the overlap is greater than or equal to a first preset threshold and less than or equal to a second preset threshold, then reflective points are selected in the centered neighborhood of the target point cloud and the source point cloud, and coarse registration is performed between the target point cloud and the source point cloud based on the selected reflective points, including: When the point cloud overlap is between 20% and 40%, the center of the reflective point is extracted by manually selecting points; Selecting reflective points in the centered neighborhood of the target point cloud and the source point cloud specifically includes: determining the centers of the target point cloud and the source point cloud respectively; using the center of the target point cloud or the center of the source point cloud as seed points, and employing a region growing algorithm based on curvature and color criteria to determine the centered neighborhood of the target point cloud and the centered neighborhood of the source point cloud respectively; and selecting multiple corresponding reflective points in the centered neighborhood of the target point cloud and the centered neighborhood of the source point cloud. Then, by registering the selected points, the rotation matrix R and translation matrix T of the point cloud are calculated. The original point cloud is then transformed by the rotation matrix R and translation matrix T to align the two point clouds.

2. The interactive registration method for house point clouds based on sparse geometric features according to claim 1, characterized in that, Reflective dots are evenly pasted on the flat surface of the object being scanned inside the building. Multi-frame point clouds of the object are collected, followed by: Downsampling is performed on the acquired point cloud data from multiple frames. An automatic filtering method is used to denoise the downsampled point cloud. Interactive point and line selection methods are used to filter out tilted and curved point cloud boundaries.

3. The interactive registration method for house point clouds based on sparse geometric features according to claim 1, characterized in that, The formula for calculating the degree of overlap is: In the formula, S represents the degree of overlap, m represents the number of overlapping reflective points between the two point clouds, and M represents the sum of the number of reflective points between the two point clouds.

4. The interactive registration method for house point clouds based on sparse geometric features according to claim 1, characterized in that, ICP fine registration includes: Construct the objective function as follows In the formula, E(R,T) represents the sum of squares of the Euclidean distances between corresponding points, R represents the rotation matrix, T represents the translation matrix, and n represents the number of selected points; Select a point p in the overlapping region of the source point cloud. i ; where p i The selected point is the i-th point; For a selected point p i Applying the rotation matrix R, we obtain p i Projection point p' in the target point cloud i ; Find the distance projection point p' in the target point cloud. i The nearest point q i As p i The corresponding points form corresponding point pairs; Based on all corresponding point pairs, with the goal of minimizing the sum of squared Euclidean distances between corresponding points, the objective function is solved iteratively using the least squares method to obtain the optimal rotation and translation matrices. The source and target point clouds are precisely registered using the optimal rotation and translation matrices to align the point clouds in adjacent frames.

5. The interactive registration method for house point clouds based on sparse geometric features according to claim 1, characterized in that, When collecting point clouds of the scanned object, it is necessary to ensure that two adjacent point clouds contain three or more corresponding points.

6. The interactive registration method for house point clouds based on sparse geometric features according to claim 2, characterized in that, The automatic filtering methods include statistical filtering and Gaussian filtering.

Images