Plane fitting-based point cloud registration method and apparatus, and electronic device

Abstract

The present application provides a plane fitting-based point cloud registration method and apparatus, and an electronic device. The method comprises: acquiring a source point cloud and a point cloud to be registered; then determining two non-parallel source planes in the source point cloud, and two non-parallel planes to be registered in the point cloud to be registered; on the basis of the two source planes and the two planes to be registered, performing calculation to obtain a rotation matrix, and obtaining a translation vector on the basis of the source point cloud and the point cloud to be registered; and finally, on the basis of the rotation matrix and the translation vector, rotating and translating the point cloud to be registered to achieve registration of the point cloud to be registered and the source point cloud. According to the present solution, registration is achieved on the basis of the correspondence between planes rather than local features of points, thereby greatly reducing the amount of calculation, and greatly reducing the sensitivity to noise.

Description

Point cloud registration method, device and electronic device based on plane fitting

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to Chinese patent application number 2023114334562 filed with the China Patent Office on November 1, 2023, entitled “Point cloud registration method, device and electronic device based on plane fitting”, the entire contents of which are incorporated by reference into this application. Technical Field

[0003] The present application relates to the field of computer technology, and in particular to a point cloud registration method, device, and electronic device based on plane fitting. Background Art

[0004] Point cloud registration is the process of matching one point cloud (the point cloud to be registered) to another point cloud (the source point cloud) based on the transformation matrix between the two point clouds (including the rotation matrix and translation vector), thereby achieving consistency in the coordinates of the two point clouds. Since the registration accuracy of point cloud registration methods directly affects the reliability of subsequent processes such as error analysis, point cloud registration is a key step in computer vision and computer-aided geometry fields such as stereo model matching, object recognition, pose estimation, and image matching.

[0005] Existing point cloud registration technology usually includes two processes: coarse registration and fine registration. The coarse registration process provides a better initial registration value for fine registration. Existing point cloud coarse registration methods generally use a one-to-one correspondence between points to achieve registration. This method used in the prior art has the problem of large computational complexity because it is necessary to calculate the features of the points in the point cloud to be registered point by point, and compare the features of these points with the features of the points in the source point cloud to determine the correct correspondence between the points. In addition, when calculating the features of a point, the method used in the prior art usually uses the point cloud information of the neighborhood near the point. When there is noise in the point cloud, it will affect the distribution of points in the neighborhood near the point, thereby affecting the calculation of the point features. Therefore, the method used in the prior art has the defects of large computational complexity and being easily affected by noise.

[0006] Application Contents

[0007] The objectives of the present application include, for example, providing a point cloud registration method, device and electronic device based on plane fitting, which can achieve registration between two point clouds and reduce the amount of calculation and sensitivity to noise.

[0008] The embodiments of the present application can be implemented as follows:

[0009] In a first aspect, the present application provides a point cloud registration method based on plane fitting, the method comprising:

[0010] Get the source point cloud and the point cloud to be registered;

[0011] Determining two source planes in the source point cloud and two to-be-registered planes in the to-be-registered point cloud, wherein the two source planes are not parallel to each other and the two to-be-registered planes are not parallel to each other;

[0012] A rotation matrix is ​​calculated based on the two source planes and the two planes to be registered, and a translation vector is obtained based on the source point cloud and the point cloud to be registered;

[0013] The point cloud to be registered is rotated and translated according to the rotation matrix and the translation vector to achieve registration of the point cloud to be registered with the source point cloud.

[0014] In an optional embodiment, the step of determining two planes to be registered in the point cloud to be registered includes:

[0015] Detecting all planes existing in the point cloud to be registered;

[0016] Determine the largest plane from all detected planes as the first plane to be registered, wherein the largest plane is the plane with the most points;

[0017] A plane that is not parallel to the first plane to be registered among all planes is determined, and a largest plane is determined from all planes that are not parallel to the first plane to be registered as a second plane to be registered.

[0018] In an optional embodiment, the step of detecting all planes existing in the point cloud to be registered includes:

[0019] Each time, three points are taken from all the points in the point cloud to be registered, and a plane passing through the three points is recorded and counted, wherein the count is increased by 1 when the planes determined by the three points are the same;

[0020] When the count of the plane to be registered reaches a preset value, it is determined that the plane has been detected, and the process continues until all planes in the point cloud to be registered are detected.

[0021] In an optional embodiment, when the count of the plane to be registered reaches a preset value, the step of determining that the plane is detected, and continuing until all planes in the point cloud to be registered are detected, includes:

[0022] When the count of the to-be-registered plane reaches a preset value, the plane is determined to be detected and the distance from each point in the to-be-registered point cloud to the plane is calculated in sequence;

[0023] Delete the points in the point cloud to be registered whose distance is less than a preset distance, and take out three points at a time from the remaining points, record the plane passing through the three points and recount the plane to continue plane detection, until the number of remaining points in the point cloud to be registered is less than a preset number, and determine that all planes in the point cloud to be registered are detected.

[0024] In an optional embodiment, the source plane includes a first source plane and a second source plane, and the planes to be registered include a first plane to be registered and a second plane to be registered;

[0025] The step of calculating a rotation matrix based on the two source planes and the two planes to be registered comprises:

[0026] Calculating a rotation matrix under a first registration mode when the first plane to be registered corresponds to the first source plane and the second plane to be registered corresponds to the second source plane;

[0027] Calculating a rotation matrix under a second registration mode when the second plane to be registered corresponds to the first source plane and the first plane to be registered corresponds to the second source plane;

[0028] A final rotation matrix is ​​determined from the rotation matrices in the first registration mode and the second registration mode.

[0029] In an optional embodiment, the step of determining a final rotation matrix from the rotation matrices in the first registration mode and the second registration mode includes:

[0030] Calculating a first error between the to-be-registered point cloud and the source point cloud after registration in the first registration method, and a second error between the to-be-registered point cloud and the source point cloud after registration in the second registration method;

[0031] The rotation matrix corresponding to the minimum error between the first error and the second error is used as the final rotation matrix.

[0032] In an optional embodiment, the step of calculating a first error between the to-be-registered point cloud and the source point cloud after registration in the first registration mode includes:

[0033] sequentially traversing each point in the point cloud to be registered after being registered using the first registration method, determining a point in the source point cloud that is closest to a point in the traversed point cloud to be registered, and calculating a distance between the closest point and the traversed point in the point cloud to be registered;

[0034] The distances corresponding to the traversed points are accumulated, and the accumulated value is divided by the total number of points in the point cloud to be registered to obtain a first error.

[0035] In an optional embodiment, the step of rotating and translating the point cloud to be registered according to the rotation matrix and the translation vector includes:

[0036] In the case of the first registration mode, obtaining normal vectors of the first plane to be registered and the first source plane respectively;

[0037] Calculating a rotation axis and a rotation angle according to the normal vector of the first plane to be registered and the normal vector of the first source plane;

[0038] The first plane to be registered and the second plane to be registered are rotated and translated according to the rotation axis, the rotation angle and the translation vector.

[0039] In a second aspect, the present application provides a point cloud registration device based on plane fitting, the device comprising:

[0040] An acquisition module configured to acquire a source point cloud and a point cloud to be registered;

[0041] a determination module configured to determine two source planes in the source point cloud and two to-be-registered planes in the to-be-registered point cloud, wherein the two source planes are not parallel to each other and the two to-be-registered planes are not parallel to each other;

[0042] a calculation module configured to calculate a rotation matrix based on the two source planes and the two planes to be registered, and obtain a translation vector based on the source point cloud and the point cloud to be registered;

[0043] The registration module is configured to rotate and translate the point cloud to be registered according to the rotation matrix and the translation vector to achieve registration between the point cloud to be registered and the source point cloud.

[0044] In a third aspect, the present application provides an electronic device, which includes a machine-readable storage medium and a processor, wherein the machine-readable storage medium stores machine-executable instructions. When the processor executes the machine-executable instructions, the electronic device implements the method described in any one of the aforementioned embodiments.

[0045] The beneficial effects of the embodiments of the present application include, for example:

[0046] The present application provides a point cloud registration method, device, and electronic device based on plane fitting. After obtaining the source point cloud and the point cloud to be registered, two non-parallel source planes in the source point cloud and two non-parallel planes to be registered in the point cloud to be registered are determined. A rotation matrix is ​​calculated based on the two source planes and the two planes to be registered, and a translation vector is obtained based on the source point cloud and the point cloud to be registered. Finally, the point cloud to be registered is rotated and translated according to the rotation matrix and the translation vector to achieve registration of the point cloud to be registered with the source point cloud. This solution achieves registration based on the correspondence between planes rather than based on the local features of the points, which can greatly reduce the amount of calculation and greatly reduce the sensitivity to noise. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without creative work.

[0048] FIG1 is a flow chart of a point cloud registration method provided in an embodiment of the present application;

[0049] FIG2 is a schematic diagram of two mutually parallel planes;

[0050] Figure 3 is a schematic diagram of the source point cloud placed in the coordinate system;

[0051] FIG4 is a flow chart of the sub-steps included in S12 in FIG1 ;

[0052] FIG5 is a flow chart of the sub-steps included in S121 in FIG4 ;

[0053] FIG6 is a flow chart of the sub-steps included in S13 in FIG1 ;

[0054] FIG7 is a flow chart of the sub-steps included in S133 in FIG6 ;

[0055] FIG8 is a flow chart of the sub-steps included in S1331 in FIG7 ;

[0056] FIG9 is a schematic diagram of the overall process of the point cloud registration method provided in an embodiment of the present application;

[0057] FIG10 is a schematic diagram of the overall flow of a plane detection method provided in an embodiment of the present application;

[0058] FIG11 is a functional module block diagram of a point cloud registration device provided in an embodiment of the present application;

[0059] FIG12 is a structural block diagram of an electronic device provided in an embodiment of the present application.

[0060] Icons: 110 - point cloud registration device based on plane fitting; 111 - acquisition module; 112 - determination module; 113 - calculation module; 114 - registration module; 120 - processor; 130 - memory; 140 - communication module. DETAILED DESCRIPTION

[0061] To make the objectives, technical solutions, and advantages of the embodiments of the present application more clear, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Generally, the components of the embodiments of the present application described and shown in the drawings herein can be arranged and designed in various different configurations.

[0062] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application for protection, but merely represents selected embodiments of the present application. All other embodiments obtained by persons of ordinary skill in the art based on the embodiments in the present application without creative work are within the scope of protection of the present application.

[0063] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, it does not need to be further defined or explained in subsequent drawings.

[0064] In the description of this application, it should be noted that the terms "first", "second", etc., if used, are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0065] It should be noted that, in the absence of conflict, the features in the embodiments of this application can be combined with each other.

[0066] Figure 1 shows a schematic flow chart of a plane-fitting-based point cloud registration method provided in an embodiment of the present application. This plane-fitting-based point cloud registration method can be performed by a point cloud registration device. This point cloud registration device can be implemented using software and / or hardware and can be configured in an electronic device, such as a computer device with relevant software installed. The detailed steps of this plane-fitting-based point cloud registration method are described below.

[0067] S11, obtaining the source point cloud and the point cloud to be registered.

[0068] S12: Determine two source planes in the source point cloud and two to-be-registered planes in the to-be-registered point cloud, wherein the two source planes are not parallel to each other and the two to-be-registered planes are not parallel to each other.

[0069] S13: Calculate a rotation matrix based on the two source planes and the two planes to be registered, and obtain a translation vector based on the source point cloud and the point cloud to be registered.

[0070] S14, rotating and translating the point cloud to be registered according to the rotation matrix and the translation vector to achieve registration of the point cloud to be registered with the source point cloud.

[0071] In this example, let's denote the source point cloud P and the point cloud to be registered Q. The source point cloud is a workpiece model point cloud constructed using computer software based on the workpiece model, while the point cloud to be registered is a point cloud composed of images of the workpiece captured using an imaging device, such as a 3D camera. Generally speaking, the source point cloud is noise-free, while the point cloud to be registered does contain noise.

[0072] Therefore, in this embodiment, after obtaining the point cloud to be registered, the point cloud to be registered can be filtered to remove some noise in the point cloud to be registered. The filtering method can adopt the method in the prior art, and this embodiment is not limited here.

[0073] Then, planes are detected in the point cloud to be registered, and two planes to be registered are determined from the detected planes. Similarly, two source planes are determined from the source point cloud. In this embodiment, the two source planes are two non-parallel planes, and the two planes to be registered are two non-parallel planes, that is, there is a certain angle between them. In this embodiment, two mutually perpendicular planes can be selected as source planes and two mutually perpendicular planes as planes to be registered.

[0074] In this embodiment, the two source planes and the two planes to be registered are not parallel to each other because, if only the correspondence between a single plane is used, the pose transformation between the source point cloud and the point cloud to be registered cannot be fully determined. For example, as shown in Figure 2, for two overlapping planes, they have the same normal vector, but the two planes can be rotated around the normal vector, so a unique pose transformation cannot be determined. Therefore, in this embodiment, two non-parallel source planes and two non-parallel planes to be registered are selected to determine the pose transformation between the source point cloud and the point cloud to be registered.

[0075] In this embodiment, a rotation matrix is ​​calculated based on the relationship between the two source planes and the two planes to be registered. For example, the rotation axis, rotation angle, etc. can be calculated through the normal vectors of the two source planes and the two planes to be registered, and then the rotation matrix is ​​obtained based on the rotation axis and rotation angle.

[0076] Furthermore, a translation vector is obtained based on the source point cloud and the point cloud to be registered. Specifically, the position of the center of mass of the source point cloud and the position of the center of mass of the point cloud to be registered can be determined. The translation vector is obtained based on the difference between the center of mass position of the source point cloud and the center of mass position of the point cloud to be registered.

[0077] On this basis, the point cloud to be registered is rotated and translated according to the rotation matrix and translation vector, so that the point cloud to be registered matches the source point cloud after rotation and translation, that is, the consistency of coordinates is achieved, thereby realizing the registration of the point cloud to be registered and the source point cloud.

[0078] In this embodiment, when performing registration, the source point cloud can be placed in a coordinate system in a certain manner. For example, the center of mass of the source point cloud can be placed at the coordinate origin, one source plane can be parallel to the xy plane, and the other source plane can be parallel to the yz plane, as shown in Figure 3. In addition, the point cloud to be registered can also be placed in a coordinate system, and registration between the two point clouds can be performed within the coordinate system.

[0079] The point cloud registration method provided in this embodiment aims to achieve coarse registration between two point clouds. Since coarse point cloud registration does not require establishing a precise correspondence between the point clouds, it only requires a rough pose transformation from the point cloud to be registered to the source point cloud, providing a good initial pose for subsequent fine registration using a fine registration algorithm. Therefore, the coarse point cloud registration method should have a low computational load and high robustness. That is, even in the presence of noise interference in the point clouds, it should still be able to determine the pose transformation between the two point clouds in a relatively short time.

[0080] The point cloud registration method provided in this embodiment does not rely on the one-to-one correspondence between points during the registration process, but uses the correspondence between planes to achieve coarse registration. Therefore, the defect of large computational complexity in existing coarse registration methods can be overcome. Moreover, since the registration is achieved based on plane detection in this embodiment rather than based on the local features of the points, the sensitivity to noise can be greatly reduced. The plane detection method relies on the overall distribution of points on the plane in the point cloud. Even if there are some noise points in certain positions in the point cloud, the local noise points will not affect the plane detection process, nor will they affect the registration process. Generally speaking, the noise in the point cloud is manifested as the fluctuation of points on the surface of the object, and these fluctuating points will not affect the plane detection process.

[0081] Therefore, the point cloud registration method based on plane fitting provided in this embodiment realizes registration based on the correspondence between planes rather than the local features of points, which can greatly reduce the amount of calculation and the sensitivity to noise.

[0082] In this embodiment, as a possible implementation, referring to FIG. 4 , the step of determining two planes to be registered in the point cloud to be registered may include the following sub-steps:

[0083] S121: Detect all planes existing in the point cloud to be registered.

[0084] S122 : Determine the largest plane from all detected planes as the first plane to be registered, wherein the largest plane is the plane with the most points.

[0085] S123 , determining a plane that is not parallel to the first plane to be registered among all planes, and determining the largest plane among all planes that are not parallel to the first plane to be registered as the second plane to be registered.

[0086] In this embodiment, each plane in the point cloud to be registered is first detected, and the plane with the most points among the detected planes is then used as the first plane to be registered. The remaining planes, excluding the first plane to be registered, that are not parallel to the first plane to be registered and that have the most points are used as the second plane to be registered. In this embodiment, specifically, the second plane to be registered can be a plane perpendicular to the first plane to be registered.

[0087] In the prior art, 3D Hough transform methods are generally used to detect planes in point clouds. The standard 3D Hough transform method selects one point from the point cloud at a time and then calculates all planes passing through that point to achieve plane detection. In this embodiment, an improved 3D Hough transform method is used to detect planes in point clouds. Please refer to Figure 5. Specifically, this can be achieved through the following methods:

[0088] S1211 , extracting three points from all points of the point cloud to be registered each time, recording a plane passing through the three points and counting the plane, wherein the count is increased by 1 when the planes determined by the three points are the same.

[0089] S1212: When the count of the to-be-registered plane reaches a preset value, it is determined that the plane is detected, until all planes in the to-be-registered point cloud are detected.

[0090] In the improved 3D Hough transform method used in this embodiment, three points are selected from the point cloud to be registered each time, and the three points are any three points in the point cloud to be registered, and a plane passing through the three points is determined. When a plane is determined, the plane can be recorded and counted, wherein an array can be used. The planes are recorded and counted using the subscript of the array A. During implementation, the point cloud to be registered can be placed in a coordinate system including an x-axis, a y-axis, and a z-axis. represents the angle between the normal vector of the plane and the z-axis, θ represents the angle between the projection of the normal vector of the plane on the xy plane and the x-axis, and ρ represents the distance from the origin of the coordinate system to the plane.

[0091] For arrays The recorded plane can be counted each time the plane is detected. The initial value of the count is 0, and the count is increased by 1 when the plane is detected.

[0092] In this embodiment, each time three points are selected from the point cloud to determine a plane, and after the planes are recorded and counted, the three selected points are placed back into the point cloud, and then three points are selected from the point cloud again to determine the plane. The plane determined by the subsequent three selected points may be the same plane as the previously determined plane. In this case, the plane count is incremented by 1.

[0093] When the count of the plane to be registered reaches a preset value, it can be understood that the plane has been determined the number of times corresponding to the preset value, and it can be determined that the plane has been detected. In this way, multiple planes existing in the point cloud to be registered can be detected.

[0094] The detection process of planes in the point cloud to be registered cannot be endless. In this embodiment, the plane detection process can be stopped when certain conditions are met. At this time, all planes in the point cloud to be registered are considered to have been detected. Specifically, this process can be implemented in the following ways:

[0095] When the count of the plane to be registered reaches a preset value, the plane is determined to be detected and the distance from each point in the point cloud to the plane is calculated in sequence. Points in the point cloud to be registered with a distance less than the preset distance are deleted, and three points are extracted from the remaining points at a time. The plane passing through the extracted three points is recorded and recounted to continue plane detection. When the number of remaining points in the point cloud to be registered is less than the preset number, all planes in the point cloud to be registered are determined to be detected.

[0096] In this embodiment, each time a plane is detected, the distance from each point in the point cloud to be registered to the plane is calculated. If the distance is less than a preset distance, that is, the corresponding point is on the plane or near the plane, the corresponding point can be deleted. Then, the detection of other planes continues from the remaining points. In this way, more and more planes are detected, and the number of remaining points in the point cloud decreases. When the number of remaining points is less than the preset number, it can be considered that all planes in the point cloud have been detected, and the plane detection process can be terminated.

[0097] In this embodiment, an improved 3D Hough transform method is used to detect planes in a point cloud. Compared with the standard 3D Hough transform method that selects one point from the point cloud each time to detect a plane, this embodiment can greatly reduce the amount of calculation.

[0098] In addition, the method for detecting the plane in the source point cloud can be the same as the method for detecting the plane in the point cloud to be registered, and the method for determining two source planes from the plane in the source point cloud is the same as the method for determining two planes to be registered from the point cloud to be registered.

[0099] In this embodiment, the two source planes determined may be a first source plane and a second source plane, and the first source plane and the second source plane may be perpendicular to each other. In addition, the first plane to be registered and the second plane to be registered may be perpendicular to each other.

[0100] Since there are generally at least two mutually perpendicular planes in the source point cloud and the point cloud to be registered, the process of selecting two mutually perpendicular source planes (the first source plane and the second source plane) from the source point cloud is feasible. When all planes in the point cloud to be registered are detected, there must be two planes to be registered (the first plane to be registered and the second plane to be registered) corresponding to the two source planes.

[0101] Since the first source plane is the largest plane in the source point cloud, the largest plane in the point cloud to be registered is selected as the first plane to be registered. The second source plane is the largest plane among the planes perpendicular to the first source plane, so the largest plane among the planes perpendicular to the first plane to be registered is selected as the second plane to be registered. This approach to plane selection is reasonable and can reduce the number of correspondence comparisons.

[0102] On this basis, the rotation matrix and translation vector can be obtained based on the two source planes and the two planes to be registered. Since there are two corresponding relationships between the two source planes and the two planes to be registered, different rotation matrices exist under different corresponding relationships. Specifically, please refer to Figure 6. The calculation process of the rotation matrix can be implemented by the following method:

[0103] S131 , calculating a rotation matrix under a first registration mode when the first plane to be registered corresponds to the first source plane and the second plane to be registered corresponds to the second source plane.

[0104] S132: Calculate a rotation matrix under a second registration mode when the second plane to be registered corresponds to the first source plane and the first plane to be registered corresponds to the second source plane.

[0105] S133: Determine a final rotation matrix from the rotation matrices in the first registration mode and the second registration mode.

[0106] In this embodiment, the two correspondences described above are: the first plane to be registered corresponds to the first source plane and the second plane to be registered corresponds to the second source plane, and the second plane to be registered corresponds to the first source plane and the first plane to be registered corresponds to the second source plane. For these two correspondences, it is necessary to determine which one is more reasonable, and the rotation matrix determined under the more reasonable correspondence is used as the final rotation matrix.

[0107] In this embodiment, the error value can be used to determine which of the above corresponding relationships is more reasonable. Specifically, referring to FIG. 7 , this can be achieved in the following manner:

[0108] S1331, calculating a first error between the to-be-registered point cloud and the source point cloud after registration in the first registration mode, and a second error between the to-be-registered point cloud and the source point cloud after registration in the second registration mode.

[0109] S1332: Use the rotation matrix corresponding to the minimum error between the first error and the second error as the final rotation matrix.

[0110] In this embodiment, assuming that registration is performed in the first registration mode, it can be achieved in the following ways:

[0111] In the first registration mode, normal vectors of the first plane to be registered and the first source plane are obtained, respectively. A rotation axis and a rotation angle are calculated based on the normal vectors of the first plane to be registered and the first source plane. The first plane to be registered and the second plane to be registered are rotated and translated based on the rotation axis, rotation angle, and translation vector.

[0112] In this embodiment, the point cloud to be registered can be rotated around its centroid, and by translating the centroid based on the translation vector, the first plane to be registered is rotated to the first source plane and the second plane to be registered is rotated to the second source plane.

[0113] Assuming that the normal vectors of the first plane to be registered and the first source plane are n1 and n2 respectively, the rotation axis is: n rot =n1×n2, the rotation angle includes θ rot,1 and θ rot,2 , where θ rot,1 =arccos(|n1·n2|),θ rot,2 =π-θ rot,1 .

[0114] The rotational registration of the point cloud to be registered is performed under the first registration method and the second registration method, respectively. The point cloud to be registered after registration can be recorded as Q'. The errors corresponding to the point cloud Q' after registration under the two registration methods are calculated respectively. In particular, referring to Figure 8, the step of calculating the first error between the point cloud to be registered after registration under the first registration method and the source point cloud can include the following sub-steps:

[0115] S13311, traverse each point in the point cloud to be registered after registration under the first registration method in turn, determine the point in the source point cloud that is closest to the point in the traversed point cloud to be registered, and calculate the distance between the closest point and the traversed point in the point cloud to be registered.

[0116] S13312: Accumulate the distances corresponding to the traversed points, and divide the accumulated value by the total number of points in the point cloud to be registered to obtain a first error.

[0117] In this embodiment, the nearest point method is used to calculate the error after registration. In addition, the second error calculation method corresponding to the point cloud to be registered after registration under the second registration method is the same as the first error calculation method mentioned above, which will not be described in detail in this embodiment.

[0118] If the first error is smaller than the second error, it indicates that the correspondence relationship under the first registration method is more reasonable. If the second error is smaller, it indicates that the correspondence relationship under the second registration method is more reasonable.

[0119] In this embodiment, the rotation matrix between corresponding planes in the registration method with smaller error is used as the final rotation matrix, and the translation between the two point cloud centroids is used as the translation vector.

[0120] In order to have a clearer understanding of the point cloud registration method provided in the embodiment of the present application, the overall implementation logic of the point cloud registration method will be introduced below in conjunction with FIG9 .

[0121] Obtain the source point cloud P and the point cloud to be registered Q.

[0122] Perform filtering on the point cloud Q. This step can filter out the noise in the point cloud Q.

[0123] Use a plane detection algorithm to detect planes in the point cloud Q.

[0124] Determine planes PQ1 and PQ2 among the planes existing in the point cloud Q. Plane PQ1 is the plane with the largest number of points in the point cloud Q, and plane PQ2 is the plane with the largest number of points among the planes perpendicular to plane PQ1.

[0125] Determine two planes PP1 and PP2 in the point cloud P. Plane PP1 is the plane with the most points in the point cloud P, and plane PP2 is the plane with the most points in the plane perpendicular to plane PP1.

[0126] Rotate PQ1 to PP1, and then rotate PQ2 to PP2, and record the error corresponding to the rotated point cloud Q as e1.

[0127] Rotate PQ1 to PP2, and then rotate PQ2 to PP1, and record the error corresponding to the rotated point cloud Q as e2.

[0128] Check whether the error e1 is smaller than the error e2. If the error e1 is smaller than the error e2, the plane correspondence relationship is determined to be: PQ1 corresponds to PP1, and PQ2 corresponds to PP2.

[0129] If the error e1 is greater than or equal to the error e2, the plane correspondence relationship is determined to be: PQ1 corresponds to PP2, and PQ2 corresponds to PP1.

[0130] The point cloud Q is rotated using the rotation axis and rotation angle under the determined plane correspondence to obtain the initial pose after coarse registration.

[0131] Please refer to FIG10 to introduce the process of using the plane detection algorithm in FIG9 to detect the plane existing in the point cloud Q.

[0132] First, it is detected whether the number of points in the point cloud Q exceeds a preset number.

[0133] If the number exceeds the preset number, three points are randomly selected from the point cloud Q.

[0134] Check whether the distance between the three selected points meets the preset conditions. This step is mainly to avoid the inability to determine the plane due to partial overlap of the three selected points.

[0135] If the distance between the three selected points meets the preset conditions, the plane passing through the three points is calculated and the plane is recorded and counted using array A. In addition, if the distance between the three selected points does not meet the preset conditions, three points are selected again from the point cloud Q. Each time the plane is detected, the count value of the plane is increased by 1. Using array To record the planes, and use the subscript of array A to count the planes.

[0136] Check whether the count of the plane exceeds a preset value. If not, continue to select three points from the point cloud Q and determine the plane passing through the three points. If it exceeds the preset value, it is determined that the plane has been detected.

[0137] Calculate the distance between the point in the point cloud Q and the detected plane.

[0138] Determine whether the calculated distance is less than the preset distance. If it is less than the preset distance, the corresponding point will be deleted. If the distance corresponding to the point is greater than or equal to the preset distance, the point will be retained.

[0139] Reset array A and return to check whether the number of points remaining in point cloud Q exceeds the preset number. The plane detection process ends when the number of points remaining in point cloud Q is less than the preset number.

[0140] The point cloud registration method provided in this embodiment achieves coarse point cloud registration through plane detection and the correspondence between the detected planes and standard planes in the source point cloud. The use of an improved 3D Hough transform for plane detection accelerates plane detection. This method avoids the computational overhead and noise sensitivity of existing traditional methods, significantly reducing computational complexity and noise sensitivity.

[0141] Based on the same inventive concept, please refer to Figure 11, which shows a functional module diagram of a point cloud registration device 110 based on plane fitting provided in an embodiment of the present application. This embodiment can divide the functional modules of the point cloud registration device 110 based on plane fitting according to the above-mentioned method embodiment. For example, each functional module can be divided according to each function, or two or more functions can be integrated into one processing module. The above-mentioned integrated module can be implemented in the form of hardware or in the form of a software functional module. It should be noted that the division of modules in the embodiment of the present application is schematic and is only a logical functional division. There may be other division methods in actual implementation.

[0142] For example, when the functional modules are divided according to their functions, the plane fitting-based point cloud registration device 110 shown in FIG11 is only a schematic diagram of the device. The plane fitting-based point cloud registration device 110 may include an acquisition module 111, a determination module 112, a calculation module 113, and a registration module 114. The functions of each functional module of the plane fitting-based point cloud registration device 110 are described in detail below.

[0143] An acquisition module 111 is configured to acquire a source point cloud and a point cloud to be registered;

[0144] It can be understood that the acquisition module 111 can be configured to execute the above step S11. For the detailed implementation of the acquisition module 111, reference can be made to the relevant content of the above step S11.

[0145] a determination module 112 configured to determine two source planes in the source point cloud and two to-be-registered planes in the to-be-registered point cloud, wherein the two source planes are not parallel to each other and the two to-be-registered planes are not parallel to each other;

[0146] It can be understood that the determination module 112 can be configured to execute the above step S12. For the detailed implementation of the determination module 112, reference can be made to the relevant content of the above step S12.

[0147] A calculation module 113 is configured to calculate a rotation matrix based on the two source planes and the two planes to be registered, and obtain a translation vector based on the source point cloud and the point cloud to be registered;

[0148] It can be understood that the calculation module 113 can be configured to execute the above step S13. For the detailed implementation of the calculation module 113, reference can be made to the relevant content of the above step S13.

[0149] The registration module 114 is configured to rotate and translate the to-be-registered point cloud according to the rotation matrix and the translation vector, so as to achieve registration between the to-be-registered point cloud and the source point cloud.

[0150] It can be understood that the registration module 114 can be configured to perform the above step S14. For the detailed implementation of the registration module 114, reference can be made to the relevant content of the above step S14.

[0151] In a possible implementation, the determining module 112 may be configured to:

[0152] Detecting all planes existing in the point cloud to be registered;

[0153] Determine the largest plane from all detected planes as the first plane to be registered, wherein the largest plane is the plane with the most points;

[0154] A plane that is not parallel to the first plane to be registered among all planes is determined, and a largest plane is determined from all planes that are not parallel to the first plane to be registered as a second plane to be registered.

[0155] In a possible implementation, the determining module 112 may be specifically configured to:

[0156] Each time, three points are taken from all the points in the point cloud to be registered, and a plane passing through the three points is recorded and counted, wherein the count is increased by 1 when the planes determined by the three points are the same;

[0157] When the count of the plane to be registered reaches a preset value, it is determined that the plane has been detected, and the process continues until all planes in the point cloud to be registered are detected.

[0158] In a possible implementation, the determining module 112 may be specifically configured to:

[0159] When the count of the to-be-registered plane reaches a preset value, the plane is determined to be detected and the distance from each point in the to-be-registered point cloud to the plane is calculated in sequence;

[0160] Delete the points in the point cloud to be registered whose distance is less than a preset distance, and take out three points at a time from the remaining points, record the plane passing through the three points and recount the plane to continue plane detection, until the number of remaining points in the point cloud to be registered is less than a preset number, and determine that all planes in the point cloud to be registered are detected.

[0161] In a possible implementation, the source plane includes a first source plane and a second source plane, the planes to be registered include a first plane to be registered and a second plane to be registered, and the calculation module 113 may be configured as follows:

[0162] Calculating a rotation matrix under a first registration mode when the first plane to be registered corresponds to the first source plane and the second plane to be registered corresponds to the second source plane;

[0163] Calculating a rotation matrix under a second registration mode when the second plane to be registered corresponds to the first source plane and the first plane to be registered corresponds to the second source plane;

[0164] A final rotation matrix is ​​determined from the rotation matrices in the first registration mode and the second registration mode.

[0165] In a possible implementation, the calculation module 113 may be specifically configured as follows:

[0166] Calculating a first error between the to-be-registered point cloud and the source point cloud after registration in the first registration method, and a second error between the to-be-registered point cloud and the source point cloud after registration in the second registration method;

[0167] The rotation matrix corresponding to the minimum error between the first error and the second error is used as the final rotation matrix.

[0168] In a possible implementation, the calculation module 113 may be specifically configured as follows:

[0169] sequentially traversing each point in the point cloud to be registered after being registered using the first registration method, determining a point in the source point cloud that is closest to a point in the traversed point cloud to be registered, and calculating a distance between the closest point and the traversed point in the point cloud to be registered;

[0170] The distances corresponding to the traversed points are accumulated, and the accumulated value is divided by the total number of points in the point cloud to be registered to obtain a first error.

[0171] In a possible implementation, the registration module 114 may be configured to:

[0172] In the case of the first registration mode, obtaining normal vectors of the first plane to be registered and the first source plane respectively;

[0173] Calculating a rotation axis and a rotation angle according to the normal vector of the first plane to be registered and the normal vector of the first source plane;

[0174] The first plane to be registered and the second plane to be registered are rotated and translated according to the rotation axis, the rotation angle and the translation vector.

[0175] Please refer to Figure 12, which is a block diagram of the structure of an electronic device provided in an embodiment of the present application. This electronic device can be the aforementioned server. The electronic device includes a memory 130, a processor 120, and a communication module 140. The memory 130, processor 120, and communication module 140 are electrically connected to each other directly or indirectly to enable data transmission or interaction. For example, these components can be electrically connected to each other via one or more communication buses or signal lines.

[0176] Memory 130 is used to store programs or data. Memory 130 may be, but is not limited to, random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), and electrically erasable programmable read-only memory (EEPROM).

[0177] The processor 120 is used to read / write data or programs stored in the memory 130 and execute the point cloud registration method based on plane fitting provided in any embodiment of the present application.

[0178] The communication module 140 is used to establish a communication connection between the electronic device and other communication terminals through a network, and to send and receive data through the network.

[0179] It should be understood that the structure shown in FIG12 is merely a schematic structural diagram of an electronic device, and the electronic device may further include more or fewer components than shown in FIG12 , or have a configuration different from that shown in FIG12 .

[0180] Furthermore, an embodiment of the present application also provides a computer-readable storage medium, which stores machine-executable instructions. When the machine-executable instructions are executed, the point cloud registration method based on plane fitting provided in the above embodiment is implemented.

[0181] Specifically, the computer-readable storage medium can be a general-purpose storage medium, such as a removable disk or hard disk. When the computer program on the computer-readable storage medium is executed, the above-described point cloud registration method based on plane fitting can be executed. Regarding the processes involved when the computer-readable storage medium and its executable instructions are executed, please refer to the relevant description in the above-mentioned method embodiment and will not be described in detail here.

[0182] In summary, the point cloud registration method, device, and electronic device based on plane fitting provided in the embodiments of the present application, after obtaining the source point cloud and the point cloud to be registered, determine two non-parallel source planes in the source point cloud and two non-parallel planes to be registered in the point cloud to be registered, calculate a rotation matrix based on the two source planes and the two planes to be registered, and obtain a translation vector based on the source point cloud and the point cloud to be registered. Finally, the point cloud to be registered is rotated and translated according to the rotation matrix and the translation vector to achieve registration of the point cloud to be registered with the source point cloud. This solution achieves registration based on the correspondence between planes, rather than based on the local features of the points. Therefore, it can greatly reduce the amount of calculation and greatly reduce the sensitivity to noise.

[0183] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A point cloud registration method based on plane fitting, characterized in that: The method comprises: Get the source point cloud and the point cloud to be registered; Determine two source planes in the source point cloud and two to-be-registered planes in the to-be-registered point cloud, wherein the two source planes are not parallel to each other and the two to-be-registered planes are not parallel to each other; A rotation matrix is ​​calculated based on the two source planes and the two planes to be registered, and a translation vector is obtained based on the source point cloud and the point cloud to be registered; The point cloud to be registered is rotated and translated according to the rotation matrix and the translation vector to achieve registration of the point cloud to be registered with the source point cloud.

2. The point cloud registration method based on plane fitting according to claim 1, characterized in that: The step of determining two planes to be registered in the point cloud to be registered comprises: Detecting all planes existing in the point cloud to be registered; Determine the largest plane from all detected planes as the first plane to be registered, wherein the largest plane is the plane with the most points; A plane that is not parallel to the first plane to be registered is determined among all planes, and a largest plane is determined from all planes that are not parallel to the first plane to be registered as a second plane to be registered.

3. The point cloud registration method based on plane fitting according to claim 2, characterized in that: The step of detecting all planes existing in the point cloud to be registered includes: Each time, three points are taken from all the points of the point cloud to be registered, and the plane passing through the three points is recorded and counted, wherein the count is increased by 1 when the planes determined by the three points are the same; When the count of the to-be-registered plane reaches a preset value, it is determined that the plane has been detected, until all planes in the to-be-registered point cloud are detected.

4. The point cloud registration method based on plane fitting according to claim 3, characterized in that: When the count of the plane to be registered reaches a preset value, the step of determining that the plane is detected, until all planes in the point cloud to be registered are detected, comprises: When the count of the to-be-registered plane reaches a preset value, it is determined that the plane is detected and the distances from each point in the to-be-registered point cloud to the plane are calculated in sequence; Delete the points in the point cloud to be registered whose distance is less than the preset distance, and take out three points at a time from the remaining points, record the plane passing through the three points taken out and recount the plane to continue plane detection, until the number of remaining points in the point cloud to be registered is less than the preset number, and determine that all planes in the point cloud to be registered are detected.

5. The point cloud registration method based on plane fitting according to claim 1, characterized in that: The source plane includes a first source plane and a second source plane, and the plane to be registered includes a first plane to be registered and a second plane to be registered; The step of calculating a rotation matrix based on the two source planes and the two planes to be registered comprises: Calculating a rotation matrix under a first registration mode when the first plane to be registered corresponds to the first source plane and the second plane to be registered corresponds to the second source plane; Calculating a rotation matrix under a second registration mode when the second plane to be registered corresponds to the first source plane and the first plane to be registered corresponds to the second source plane; A final rotation matrix is ​​determined from the rotation matrices in the first registration mode and the second registration mode.

6. The point cloud registration method based on plane fitting according to claim 5, characterized in that: The step of determining a final rotation matrix from the rotation matrices in the first registration mode and the second registration mode comprises: Calculating a first error between the to-be-registered point cloud and the source point cloud after being registered in the first registration mode, and a second error between the to-be-registered point cloud and the source point cloud after being registered in the second registration mode; The rotation matrix corresponding to the minimum error between the first error and the second error is used as the final rotation matrix.

7. The point cloud registration method based on plane fitting according to claim 6, characterized in that: The step of calculating a first error between the to-be-registered point cloud and the source point cloud after registration in the first registration mode comprises: Traversing each point in the point cloud to be registered after being registered in the first registration method in sequence, determining the point in the source point cloud that is closest to the point in the traversed point cloud to be registered, and calculating the distance between the point that is closest to the traversed point in the point cloud to be registered; The distances corresponding to the traversed points are accumulated, and the accumulated value is divided by the total number of points in the point cloud to be registered to obtain a first error.

8. The point cloud registration method based on plane fitting according to claim 5, characterized in that: The step of rotating and translating the point cloud to be registered according to the rotation matrix and the translation vector comprises: In the case of the first registration mode, obtaining normal vectors of the first plane to be registered and the first source plane respectively; Calculating a rotation axis and a rotation angle according to the normal vector of the first plane to be registered and the normal vector of the first source plane; The first plane to be registered and the second plane to be registered are rotated and translated according to the rotation axis, the rotation angle and the translation vector.

9. A point cloud registration device based on plane fitting, characterized in that: The device comprises: An acquisition module, configured to acquire a source point cloud and a point cloud to be registered; A determination module, configured to determine two source planes in the source point cloud and two to-be-registered planes in the to-be-registered point cloud, wherein the two source planes are not parallel to each other and the two to-be-registered planes are not parallel to each other; A calculation module, configured to calculate a rotation matrix based on the two source planes and the two planes to be registered, and obtain a translation vector based on the source point cloud and the point cloud to be registered; The registration module is configured to rotate and translate the point cloud to be registered according to the rotation matrix and the translation vector to achieve registration of the point cloud to be registered with the source point cloud.

10. An electronic device, characterized in that: The electronic device includes a machine-readable storage medium and a processor, wherein the machine-readable storage medium stores machine-executable instructions, and when the processor executes the machine-executable instructions, the electronic device implements the method described in any one of claims 1 to 8.

Images