Building indoor modeling point cloud registration method based on key point extraction

By using key point extraction and gradient descent methods to register point clouds in the building interior, the problem of low registration accuracy of three-dimensional model in the building interior in the prior art is solved, and high-precision point clouds are achieved in complex or simple scenarios Registration and 3D model acquisition.

CN119006553AInactive Publication Date: 2024-11-22XIAN CNNC NUCLEAR INSTRUMENT CO LTD +2
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
CN202411495409.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-10-25
Publication Date
2024-11-22
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

The accuracy of the existing three-dimensional indoor model registration method in complex or simple geometric structure environments is reduced, and an accurate three-dimensional indoor model cannot be obtained, especially when there are few feature points on newly built concrete or putty walls and poor depth map accuracy. .

Method used

Using the indoor modeling point cloud registration method of building interior modeling based on key point extraction, the depth map is taken through a three-dimensional camera, the key point extraction algorithm (such as FAST and ORB) is used to extract key points, pair and restore it to point cloud, and use gradient descent method The random minimum value search method solves the rotation and translation variables, and establishes a loss function for registration.

Benefits of technology

Point cloud registration is completed with higher accuracy in complex or simple indoor scenes of buildings, which significantly improves registration accuracy, reduces the amount of calculation, and obtains a more accurate indoor three-dimensional model.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119006553A_ABST
    Figure CN119006553A_ABST
Patent Text Reader

Abstract

The invention discloses a building indoor modeling point cloud registration method based on key point extraction. The method comprises the steps of obtaining a building indoor depth map, extracting key points of the depth map, restoring the key points into point clouds, registering the point clouds of the key points, registering all the point clouds and the like. According to the method, the key points are extracted from the shot source depth map and the shot target depth map, the registration process of all the points is determined through registration of the key points, and therefore registration of the point cloud of the whole source depth map and the point cloud of the target depth map is achieved. According to the method provided by the invention, the point cloud registration of the indoor environment can be completed with relatively high precision in a building indoor scene with a particularly complex geometric structure or a particularly simple geometric structure, so that an accurate indoor three-dimensional model is obtained, and the registration precision is obviously higher than that of a common Globe + ICP registration method.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to the technical field of three-dimensional reconstruction of objects, and in particular to a point cloud registration method for indoor modeling of buildings based on key point extraction. Background Art

[0002] As the main technical equipment for intelligent construction, construction robots have been able to perform autonomous operations such as engineering measurement, quality inspection, polishing, spraying, marking, and drilling, and have been widely used in construction projects.

[0003] Before construction work, construction robots usually need to complete task planning, operation planning, path planning, navigation positioning, obstacle avoidance and real-time control, all of which require accurate three-dimensional modeling of the building's indoor environment, which is necessary to provide data for the construction robot to perform autonomous operations. At present, the three-dimensional reconstruction of the building's indoor structure model is mainly achieved through the registration and splicing of graphics, images or three-dimensional point clouds. That is, during the movement of the robot, the sensor collects the image or depth information of the indoor environment in real time to obtain high-quality depth maps or point cloud data, and then uses filtering algorithms and downsampling algorithms to simplify these point cloud data. After initial registration and precise registration, a complete three-dimensional point cloud scene or three-dimensional model of the spliced ​​indoor environment is obtained. Therefore, using a registration method with high accuracy and high speed is the key to obtaining an accurate indoor model.

[0004] Registration is to find the transformation relationship between the two point sets in space given two 3D data points from different coordinate systems so that the two point sets can be unified into the same coordinate system. Currently, the global registration method (referred to as the Global Registration Method) and the iterative closest point (ICP) registration method are commonly used. The Global Registration Method first calculates the FPFH (Fast Point Feature Histogram) feature vectors of the two point sets, and then finds several key points in the two point sets that are close to the FPFH feature vectors, and solves the final transformation matrix by minimizing the loss function. This method is computationally intensive and not very accurate.

[0005] ICP registration is an iterative method that requires repeatedly calculating the corresponding points of each point in the source point cloud in the target point cloud, and then calculating the transformation matrix that minimizes the sum of the geometric distances between all corresponding points. This method has the advantage of high accuracy and is often used in point cloud registration. However, the disadvantage of ICP registration is that the algorithm may converge to a local optimal solution, noise or abnormal data may cause the algorithm to fail to converge, and the algorithm needs to determine the initial value of a transformation matrix in the first step of the iteration, which has a great influence on the final registration result. If the initial value is not chosen appropriately, the algorithm may be limited to a local optimal solution and cannot converge to the correct transformation matrix.

[0006] Therefore, in actual use, the global registration method is often used to obtain the initial matrix, and then the ICP method is used for precise registration. We call this method the global ICP registration method, or simply the "Global+ICP" registration method. Since this method uses both the global registration method and the ICP method, although it improves the accuracy of the 3D model compared to the global registration method, it still involves a lot of calculations.

[0007] However, the indoor environment of a building is usually complex. For example, the geometric structure of the building is complex, building materials and equipment are piled on the ground, holes, grooves, beams are reserved on the wall, and there are cables, wires, steel bars, etc. There are many corners, curves, and broken lines. The registration accuracy of existing methods (including the "Global+ICP" registration method) is significantly reduced, and it is impossible to obtain an accurate indoor 3D model. At the same time, due to the very few feature points on the newly built indoor concrete or putty walls, the poor accuracy of the depth map or point cloud, or the small overlap between the pictures, it is impossible to obtain an accurate indoor 3D model. Summary of the invention

[0008] The technical problem to be solved by the present invention is to provide a point cloud registration method for indoor modeling of buildings based on key point extraction in view of the deficiencies in the above-mentioned prior art, which can complete the point cloud registration of the indoor environment with high accuracy in indoor scenes of buildings with particularly complex or simple geometric structures, thereby obtaining an accurate indoor three-dimensional model.

[0009] In order to solve the above technical problems, the technical solution adopted by the present invention is: a point cloud registration method for indoor building modeling based on key point extraction, comprising the following steps.

[0010] Step 1: Obtain a depth map of the building interior.

[0011] A 3D camera is used to shoot indoors in a building to obtain a source depth map and a target depth map, respectively. The source depth map and the target depth map have overlapping parts.

[0012] Step 2: Extract key points of the depth map.

[0013] A keypoint extraction algorithm is used to extract the keypoints of each source depth map and the keypoints of each target depth map respectively.

[0014] Step 3: Restore key points to point cloud.

[0015] S3-1: Based on the key points of the source depth map and the key points of the target depth map, the key points of the source depth map and the key points of the target depth map with the same actual position are obtained through a pairing algorithm.

[0016] S3-2: Restore the key points of the paired source depth map and the key points of the target depth map into point clouds respectively, and obtain the key point cloud of the source depth map and the key point cloud of the target depth map.

[0017] Step 4: Register the point cloud of key points to obtain decision variables.

[0018] S4-1: Setting decision variables to be solved, the decision variables include: , m , n , p ,in, is the angle of rotation around the X-axis of the point cloud coordinate system, is the angle of rotation around the Y axis of the point cloud coordinate system, is the angle of rotation around the Z axis of the point cloud coordinate system, m is the distance of translation along the X-axis of the point cloud coordinate system, n is the distance of translation along the Y axis of the point cloud coordinate system, p It is the distance of translation along the Z axis of the point cloud coordinate system.

[0019] S4-2: Register the point cloud of the key points of the source depth map using formula (1): (1) In formula (1), is the source depth map after registration The point cloud coordinates of the key points. is the first source depth map before registration The point cloud coordinates of the key points.

[0020] Based on the key point cloud of each registered source depth map and the key point cloud of the corresponding target depth map, a loss function as shown in formula (2) is established: (2) In formula (2), is the source depth map The coordinates of the key points after point cloud registration, is the source depth map The key point cloud coordinates of the target depth map with the same actual position of the key points, h is the total number of key points, J is the loss value.

[0021] The gradient descent method is used to solve equations (1) and (2) simultaneously until the loss function converges to obtain the decision variables.

[0022] Step 5: Align all point clouds.

[0023] S5-1: restore the source depth map to a 3D point cloud, and restore the target depth map to a 3D point cloud.

[0024] S5-2: Based on the decision variables, all point clouds of the source depth map are registered by equation (1), and the three-dimensional point cloud of the target depth map is added to obtain the three-dimensional point cloud of the building interior.

[0025] Further, in step 2, the key point extraction algorithm is any one of the following: FAST target algorithm and ORB algorithm.

[0026] Furthermore, in step S3-1, the pairing algorithm is a rule-based algorithm with the minimum Euclidean distance of BRIEF descriptors.

[0027] Further, in step S3-2, the key points of the paired source depth map and the key points of the target depth map are restored into point clouds respectively, including the following steps.

[0028] S3-2-1: Get the coordinates of the key points in the source depth map or target depth map ( u , v ), and the depth value of the keypoint in the source depth map or the target depth map depth .

[0029] S3-2-2: Get camera parameters u 0 , v 0 , f x , f y , and set the parameters scale ,in,( u 0 , v 0 ) is the position of the center point of the source depth map or the target depth map, f x From the source depth map or target depth map to the camera imaging plane x The distance between the axes, f y From the source depth map or target depth map to the camera imaging plane y Axis distance.

[0030] S3-2-3: Calculate the point cloud coordinates of the key points of the source depth map or the target depth map according to formula (3) x , y , z ).

[0031] (3) In the formula, scale is the parameter to be set, (u 0 , v 0 ) is the position of the center point of the source depth map or the target depth map, f x From the source depth map or target depth map to the camera imaging plane x The distance between the axes, f y From the source depth map or target depth map to the camera imaging plane y Axis distance.

[0032] Compared with the prior art, the present invention has the following advantages: The present invention provides a point cloud registration method for indoor building modeling based on key point extraction, which extracts key points from the source depth map and the target depth map, and realizes the registration process of all points through the registration of key points, that is, the point cloud of the entire source depth map and the point cloud of the target depth map are registered, so that the point cloud registration of the indoor environment can be completed with high accuracy and low calculation amount in the indoor scene of the building with particularly complex or simple geometric structure, and an accurate indoor three-dimensional model can be obtained. The registration accuracy of this method is significantly higher than that of the commonly used "Global+ICP" registration method.

[0033] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 A schematic diagram of the overall process of a point cloud registration method for indoor building modeling based on key point extraction provided by the present invention. DETAILED DESCRIPTION

[0035] like Figure 1 As shown, the present invention provides a point cloud registration method for indoor building modeling based on key point extraction, which includes the following steps.

[0036] Step 1: Obtain a depth map of the building interior.

[0037] A 3D camera is used to shoot indoors in a building to obtain a source depth map and a target depth map, respectively. The source depth map and the target depth map have overlapping parts.

[0038] The source depth map and the target depth map are both obtained by taking pictures with a 3D camera. Since a 3D camera can only take pictures of part of the content, and a panoramic 3D image requires a depth map image that surrounds the whole scene, the images of the building interior taken by the 3D camera must be spliced ​​together, and the process of splicing is called registration. The present invention splices the source depth map to the target depth map, that is, when executing, the source depth map is adjusted based on the target depth map to obtain a spliced ​​image. The overlapped part is to determine the adjustment method, that is, the registration method.

[0039] Because the present invention is a three-dimensional stitching process, the source depth map is registered according to the target depth map, so the three-dimensional point cloud corresponding to the source depth map will change according to the registration, and after the change, plus the three-dimensional point cloud of the target depth map, they are all the three-dimensional point clouds of the entire building interior.

[0040] If the image is captured in a 3D point cloud format rather than a depth map format, it is converted into a depth map format respectively. That is, in step 1, the present invention uses a 3D camera to shoot indoors in a building to obtain a source depth map and a target depth map respectively, including the following steps.

[0041] S1-1: Use a 3D camera to shoot indoors in a building to obtain source 3D point cloud and target 3D point cloud.

[0042] S1-2: Convert the captured source 3D point cloud into a source depth map, and convert the captured target 3D point cloud into a target depth map.

[0043] When implementing, a 3D camera can be used to shoot depth maps (source depth map and target depth map) or 3D point clouds of the indoor environment to be modeled. If a 3D point cloud is shot, it is converted into a depth map, and the one-to-one correspondence between pixels and point clouds is recorded. The image is filtered, de-discrete points are removed, and downsampled.

[0044] Step 2: Extract key points of the depth map.

[0045] A keypoint extraction algorithm is used to extract the keypoints of each source depth map and the keypoints of each target depth map respectively.

[0046] In step 2 of the present invention, the key point extraction algorithm is any one of the following: FAST target algorithm and ORB algorithm. ORB (Oriented FAST and Rotated BRIEF) is a fast key point extraction algorithm, wherein the feature point extraction adopts the FAST (Features from Accelerated Segment Test) algorithm. ORB is used for the registration of two-dimensional images and has the advantages of fast running speed, accurate extraction of key points, high matching accuracy of corresponding key points, and robustness.

[0047] During implementation, the present invention uses the ORB method to extract the key points of the two acquired depth maps. First, the FAST method is used to extract the corner points, which are the key points, that is, the points with large changes in grayscale values. The principle is to first select a certain pixel point and check the circle with a length of 16 pixels around it. If there are several consecutive pixel grayscale values ​​higher or lower than the grayscale value of the selected point by more than a certain threshold among these 16 pixels, then the pixel point is the corner point.

[0048] Step 3: Restore key points to point cloud.

[0049] S3-1: Based on the key points of the source depth map and the key points of the target depth map, a pairing algorithm is used to obtain the correspondence between the key points of the source depth map and the key points of the target depth map with the same position.

[0050] In step S3-1 of the present invention, the pairing algorithm is a rule algorithm with the minimum Euclidean distance of the BRIEF descriptor. The present invention uses the BRIEF descriptor to describe the features of each key point.

[0051] The key points extracted from the two depth maps are paired in sequence according to the rule of the minimum Euclidean distance of the BRIEF descriptor to obtain several groups of paired key points. The present invention selects the point cloud of 10 pairs of key points with the minimum Euclidean distance for registration. The factor that has the greatest impact on the accuracy of point cloud registration is whether the corresponding relationship between the key points of the two images is correct.

[0052] S3-2: Restore the key points of the paired source depth map and the key points of the target depth map into point clouds respectively, and obtain the key point cloud of the source depth map and the key point cloud of the target depth map.

[0053] In the implementation process, if the depth map is converted from a point cloud, the point in the point cloud corresponding to the key point found in the depth map is obtained according to the one-to-one correspondence between pixels saved during the conversion. If the depth map is directly taken by a camera, the key point found in the depth map is restored to a point cloud according to the following method (S3-2-1 to S3-2-3).

[0054] In step S3-2, the key points of the paired source depth map and the key points of the target depth map are restored into point clouds respectively, including the following steps.

[0055] S3-2-1: Get the coordinates of the key points in the source depth map or target depth map ( u , v ), and the depth value of the keypoint in the source depth map or the target depth map depth .

[0056] S3-2-2: Get camera parametersu 0 , v 0 , f x , f y , and set the parameters scale ,in,( u 0 , v 0 ) is the position of the center point of the source depth map or the target depth map, f x From the source depth map or target depth map to the camera imaging plane x The distance between the axes, f y From the source depth map or target depth map to the camera imaging plane y Axis distance.

[0057] S3-2-3: Calculate the point cloud coordinates of the key points of the source depth map or the target depth map according to formula (3) x , y , z ).

[0058] (3) In the formula, scale is the parameter to be set, ( u 0 , v 0 ) is the position of the center point of the source depth map or the target depth map, f x From the source depth map or target depth map to the camera imaging plane x The distance between the axes, f y From the source depth map or target depth map to the camera imaging plane y Axis distance.

[0059] In general, f x and f y Equal, the same value.

[0060] Step 4: Register the point cloud of key points to obtain decision variables.

[0061] S4-1: Setting decision variables to be solved, the decision variables include: , m , n , p ,in, is the angle of rotation around the X-axis of the point cloud coordinate system, is the angle of rotation around the Y axis of the point cloud coordinate system, is the angle of rotation around the Z axis of the point cloud coordinate system, m is the distance of translation along the X-axis of the point cloud coordinate system, n is the distance of translation along the Y axis of the point cloud coordinate system, p It is the distance of translation along the Z axis of the point cloud coordinate system.

[0062] S4-2: Register the point cloud of the key points of the source depth map using formula (1): (1) In formula (1), is the source depth map after registration The point cloud coordinates of the key points. is the first source depth map before registration The point cloud coordinates of the key points.

[0063] Based on the key point cloud of each registered source depth map and the key point cloud of the corresponding target depth map, a loss function as shown in formula (2) is established.

[0064] (2) In formula (2), is the source depth map The coordinates of the key points after point cloud registration are , is the source depth map The key point cloud coordinates of the target depth map with the same actual position of the key points, h is the total number of key points, J is the loss value.

[0065] The gradient descent method is used to solve equations (1) and (2) simultaneously until the loss function converges to obtain the decision variables.

[0066] In the implementation, the gradient descent method is used to find the value of the decision variable that minimizes the objective function (2). Since the gradient descent method is prone to fall into the local optimal solution, the random minimum search method is used to find the initial descent position. The principle of the random minimum search method is to put ( m , n , p ) is fixed as the difference between the center of gravity of the target point cloud key point and the center of gravity of the source point cloud key point. Variables are searched randomly.

[0067] Step 5: Align all point clouds.

[0068] S5-1: restore the source depth map to a 3D point cloud, and restore the target depth map to a 3D point cloud.

[0069] S5-2: Based on the decision variables, all point clouds of the source depth map are registered by equation (1), and the three-dimensional point cloud of the target depth map is added to obtain the three-dimensional point cloud of the building interior.

[0070] The present invention extracts the corresponding key points in the two-dimensional depth map through the key point extraction method. The key points in the two-dimensional depth map are restored to the key points in the three-dimensional point cloud. The key points in the source point cloud and the target point cloud are aligned using the random minimum search method and the gradient descent method to obtain the final accurate indoor three-dimensional model. Among them, the two-dimensional depth map can be a depth map directly taken by a three-dimensional camera, or the point cloud can be converted into a depth map by using the equivalent conversion relationship between the three-dimensional point cloud and the depth map. After the key points are extracted using the depth map, it is converted into a point cloud.

[0071] Compared with the "global ICP" method that uses the Global method to obtain the initial matrix and then uses the ICP method for precise alignment, the present invention can complete the point cloud alignment of the indoor environment with higher accuracy in indoor scenes of buildings with particularly complex or simple geometric structures, thereby obtaining an accurate indoor three-dimensional model.

[0072] In summary, the present invention realizes the registration of the point cloud of the entire source depth map and the point cloud of the target depth map by extracting key points from the source depth map and the target depth map, and determining the registration process of all points through the registration of key points. This method can complete the point cloud registration of the indoor environment with high accuracy in indoor scenes of buildings with particularly complex or simple geometric structures, and obtain an accurate indoor three-dimensional model. The registration accuracy is significantly higher than that of the "Global+ICP" registration method.

[0073] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any way. Any simple modification, change and equivalent structural change made to the above embodiment based on the technical essence of the present invention still falls within the protection scope of the technical solution of the present invention.

Claims

1. A point cloud registration method for indoor building modeling based on key point extraction, characterized in that: It includes the following steps: Step 1: Obtain a depth map of the building interior Use a 3D camera to shoot indoors in a building to obtain a source depth map and a target depth map, respectively, and there is an overlap between the source depth map and the target depth map; Step 2: Extract key points of the depth map Use a key point extraction algorithm to extract the key points of each source depth map and the key points of each target depth map respectively; Step 3: Restore key points to point cloud S3-1: Based on the key points of the source depth map and the key points of the target depth map, the key points of the source depth map and the key points of the target depth map with the same actual position are obtained through a pairing algorithm; S3-2: restore the key points of the paired source depth map and the key points of the target depth map into point clouds respectively, to obtain the key point cloud of the source depth map and the key point cloud of the target depth map; Step 4: Register the point cloud of key points to obtain decision variables S4-1: Setting decision variables to be solved, the decision variables include: , m , n , p ,in, is the angle of rotation around the X-axis of the point cloud coordinate system, is the angle of rotation around the Y axis of the point cloud coordinate system, is the angle of rotation around the Z axis of the point cloud coordinate system, m is the distance of translation along the X axis of the point cloud coordinate system, n is the distance of translation along the Y axis of the point cloud coordinate system, p is the distance of translation along the Z axis of the point cloud coordinate system; S4-2: Register the point cloud of the key points of the source depth map using formula (1): (1) In formula (1), is the source depth map after registration Point cloud coordinates of key points; is the first source depth map before registration Point cloud coordinates of key points; Based on the key point cloud of each registered source depth map and the key point cloud of the corresponding target depth map, a loss function as shown in formula (2) is established: (2) In formula (2), is the source depth map The coordinates of the key points after point cloud registration, is the source depth map The key point cloud coordinates of the target depth map with the same actual position of the key points, h is the total number of key points, J is the loss value; Combine equations (1) and (2) and use the gradient descent method to solve until the loss function converges to obtain the decision variables; Step 5: All point clouds are aligned S5-1: restore the source depth map to a three-dimensional point cloud, and restore the target depth map to a three-dimensional point cloud; S5-2: Based on the decision variables, all point clouds of the source depth map are registered by equation (1), and the three-dimensional point cloud of the target depth map is added to obtain the three-dimensional point cloud of the building interior.

2. A point cloud registration method for indoor building modeling based on key point extraction according to claim 1, characterized in that: In step 2, the key point extraction algorithm is any one of the following: FAST target algorithm and ORB algorithm.

3. A point cloud registration method for indoor building modeling based on key point extraction according to claim 1, characterized in that: In step S3-1, the pairing algorithm is a rule-based algorithm with the minimum Euclidean distance of BRIEF descriptors.

4. A point cloud registration method for indoor building modeling based on key point extraction according to claim 1, characterized in that: In step S3-2, the key points of the paired source depth map and the key points of the target depth map are restored into point clouds respectively, including the following steps: S3-2-1: Get the coordinates of the key points in the source depth map or target depth map ( u , v ), and the depth value of the keypoint in the source depth map or the target depth map depth ; S3-2-2: Get the camera parameters u 0, v 0, f x , f y , and set the parameters scale ,in,( u 0, v 0) is the position of the center point of the source depth map or the target depth map, f x From the source depth map or target depth map to the camera imaging plane x The distance between the axes, f y From the source depth map or target depth map to the camera imaging plane y The distance between the axes; S3-2-3: Calculate the point cloud coordinates of the key points of the source depth map or the target depth map according to formula (3) x , y , z ): (3) In the formula, scale is the parameter to be set, ( u 0, v 0) is the position of the center point of the source depth map or the target depth map, f x From the source depth map or target depth map to the camera imaging plane x The distance between the axes, f y From the source depth map or target depth map to the camera imaging plane y Axis distance.

Citation Information

Patent Citations

  • Point cloud gridding method, device and equipment and computer storage medium

    CN109658515A

  • Glue path guiding and positioning method based on shape coarse registration and ICP point cloud fine registration

    CN114202566A

  • Point cloud registration method and device and electronic equipment

    CN117611643A

  • Point cloud coarse registration method, device and equipment

    CN117765039A

  • METHOD FOR ESTIMATING THE POSITIONING OF A CAMERA IN THE REFERENCE FRAME OF A THREE-DIMENSIONAL SCENE, DEVICE, AUGMENTED REALITY SYSTEM AND ASSOCIATED COMPUTER PROGRAM

    FR3073311A1