High-precision map point cloud registration method and device, electronic equipment and storage medium

Abstract

The present disclosure provides a point cloud registration method, device, electronic equipment and storage medium, relates to the technical field of automatic driving and high-precision map, and in particular to the field of point cloud data registration. The specific implementation scheme of the point cloud registration method is as follows: a plurality of groups of point clouds to be matched are acquired, and the correlation degrees of two point clouds in the plurality of groups of point clouds are determined; according to the correlation degrees, an associated point cloud pair is constructed; wherein each associated point cloud pair comprises two groups of point clouds; for each associated point cloud pair, the relative poses of the two groups of point clouds included in the associated point cloud pair are calculated, and the initial poses of the two groups of point clouds are optimized according to the relative poses; and multi-view point cloud registration is performed on the plurality of groups of point clouds based on the optimized initial poses.

Description

Technical Field

[0001] This disclosure relates to the fields of autonomous driving and high-precision mapping, and particularly to the field of point cloud data registration. Background Technology

[0002] Point cloud registration is a crucial step in high-precision map production, used to align point cloud data collected at different times and locations. Currently, point cloud registration methods mainly include pair-wise registration and multi-view registration. However, when there are a large number of overlapping point clouds across different acquisition views, pair-wise registration algorithms have high time complexity, resulting in low mapping efficiency. While multi-view registration algorithms can register multiple sets of point clouds simultaneously, improving efficiency, their accuracy remains low. Summary of the Invention

[0003] This disclosure provides a high-precision map point cloud registration method, apparatus, electronic device, and storage medium.

[0004] According to a first aspect of this disclosure, a high-precision map point cloud registration method is provided, comprising:

[0005] Obtain multiple sets of point clouds to be matched, and determine the correlation between pairs of point clouds in the multiple sets of point clouds;

[0006] Based on the correlation, associated point cloud pairs are constructed; wherein each associated point cloud pair contains two sets of point clouds;

[0007] For each associated point cloud pair, calculate the relative pose of the two sets of point clouds contained in the associated point cloud pair, and optimize the initial pose of the two sets of point clouds based on the relative pose.

[0008] Based on the optimized initial pose, multi-view point cloud registration is performed on the multiple sets of point clouds.

[0009] According to a second aspect of this disclosure, a high-precision map point cloud registration device is provided, comprising:

[0010] The determination module is used to acquire multiple sets of point clouds to be matched and determine the correlation between pairs of point clouds in the multiple sets of point clouds.

[0011] A construction module is used to construct associated point cloud pairs based on the correlation; wherein each associated point cloud pair contains two sets of point clouds;

[0012] An optimization module is used to calculate the relative pose of the two sets of point clouds contained in each associated point cloud pair, and optimize the initial pose of the two sets of point clouds based on the relative pose.

[0013] The registration module is used to perform multi-view point cloud registration on the multiple sets of point clouds based on the optimized initial pose.

[0014] According to a third aspect of this disclosure, an electronic device is provided, comprising:

[0015] At least one processor; and

[0016] A memory communicatively connected to the at least one processor; wherein,

[0017] The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method described in the first aspect.

[0018] According to a fourth aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions, wherein the computer instructions are configured to cause the computer to perform the method according to the first aspect.

[0019] According to a fifth aspect of this disclosure, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the method according to the first aspect.

[0020] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0021] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0022] Figure 1 This is a flowchart illustrating an exemplary embodiment of a high-precision map point cloud registration method.

[0023] Figure 2 This disclosure provides a flowchart of a high-precision map point cloud registration method for determining correlation, as an exemplary embodiment of the present disclosure.

[0024] Figure 3 This is a schematic diagram illustrating the effect of determining key point pairs in a high-precision map point cloud registration method provided by an exemplary embodiment of the present disclosure;

[0025] Figure 4 This disclosure provides a flowchart of a high-precision map point cloud registration method for determining relative pose, as an exemplary embodiment.

[0026] Figure 5 This is a flowchart illustrating the preprocessing of point clouds in a high-precision map point cloud registration method provided by an exemplary embodiment of the present disclosure;

[0027] Figure 6This is a schematic diagram of a high-precision map point cloud registration device provided in an exemplary embodiment of the present disclosure;

[0028] Figure 7 This is a schematic diagram of the structure of an electronic device provided by an exemplary embodiment of the present disclosure. Detailed Implementation

[0029] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0030] Currently, when dealing with a large number of overlapping point clouds from different acquisition perspectives, multi-view registration algorithms are generally used to improve registration efficiency. However, multi-view registration algorithms require high accuracy of the initial pose of the point cloud. If the accuracy of the initial pose is not high enough, fast and accurate point cloud registration cannot be achieved.

[0031] To address the aforementioned issues, this disclosure provides a high-precision map point cloud registration method to improve the accuracy of multi-view registration. This point cloud registration can be used for high-precision map creation.

[0032] Figure 1 A flowchart of a high-precision map point cloud registration method provided for an exemplary embodiment of this disclosure, the point cloud registration method including the following steps:

[0033] Step 101: Obtain multiple sets of point clouds to be matched, and determine the correlation between pairs of point clouds in the multiple sets of point clouds.

[0034] These multiple point clouds are point clouds acquired from different viewpoints. Multiple point clouds, or multiple point clouds in general, consist of a set of points, and each point cloud is represented by a set of point cloud data. During point cloud acquisition, the Global Positioning System (GPS) information of the point cloud acquisition device (e.g., LiDAR) can be recorded. Based on this GPS information, the initial pose of each set of point clouds can be determined. A single scan by the point cloud acquisition device yields one frame of point cloud data; multiple scans from different viewpoints result in multiple frames of point cloud data. A set of point clouds can be obtained by stitching together multiple frames of point clouds from the same viewpoint.

[0035] In one embodiment, the correlation between any two point clouds in multiple point cloud sets is calculated. Taking N point cloud sets as an example, the correlation between any two point clouds needs to be calculated once, requiring a total of [number missing] calculations. Calculation of secondary correlation.

[0036] In one embodiment, the correlation between pairs of point clouds in multiple point clouds is calculated. Specifically, the correlation between pairs of point clouds whose distance is less than a second distance threshold is calculated. It is understood that the farther apart two point clouds are, the lower their correlation. To reduce computational load, the correlation between pairs of point clouds whose distance is less than the second distance threshold can be calculated.

[0037] For example, if there are 6 point clouds, namely point cloud a to point cloud f, for point cloud a, the point clouds whose distance to it is less than the second distance threshold include point cloud b and point cloud f. That is, point cloud b and point cloud f are closer to point cloud a, while point clouds c to point cloud e are farther away from point cloud a. Then, we only need to calculate the correlation between point cloud a and point cloud b and point cloud f, and there is no need to calculate the similarity between point cloud a and point clouds c to point cloud e.

[0038] In one embodiment, the relevance is determined based on similarity. Specifically, point cloud features are extracted from two sets of point clouds, the similarity of the point cloud features between the two sets of point clouds is calculated, and the relevance is determined based on this similarity. Similarity and relevance are positively correlated. Point cloud features may include, but are not limited to, point cloud size, point cloud edge shape, etc.

[0039] In one embodiment, the relevance is determined based on the degree of overlap and the sufficiency of geometric constraints, see [link to relevant documentation]. Figure 2 Determining relevance involves the following steps:

[0040] Step 101-1: Unify the initial poses of the two sets of point clouds into the target coordinate system.

[0041] The target coordinate system can be the coordinate system of either of the two point clouds, or a coordinate system can be selected according to actual needs. This disclosure does not impose any particular limitation on this.

[0042] Step 101-2: Determine the overlap between the two sets of point clouds in the target coordinate system.

[0043] In one embodiment, the overlap between the two point clouds is determined based on the number of target point pairs contained in the two point clouds. A target point pair is defined as two points in the target coordinate system that belong to the two point clouds respectively and are less than a first distance threshold; the overlap is positively correlated with the number of target point pairs.

[0044] Below, we take two point clouds as point cloud P. j And point cloud P i The target coordinate system is the point cloud P. i Taking the coordinate system as an example, the implementation method for determining the degree of overlap will be further explained:

[0045] Based on the initial pose, the point cloud P j Convert to point cloud P iIn the coordinate system, the point cloud P is obtained. j ', point cloud P j 'and point cloud P i Located in the same coordinate system. Traverse the point cloud P j Each point in ', and in the point cloud P i Search for the nearest point in the search; if the distance to the nearest point is less than a first distance threshold, the two points are considered to overlap, and P is set to... j The point in ' and P i The nearest point in the point cloud is identified as the target point pair; if the distance to the nearest point is greater than or equal to a first distance threshold, the two points are considered non-overlapping and not a target point pair. The more target point pairs there are, the better the point cloud P is. j 'and point cloud P i The higher the degree of overlap, the higher the degree of overlap between the two sets of point clouds can be. Therefore, the degree of overlap between the two sets of point clouds can be determined based on the number of target point pairs.

[0046] Step 101-3: Determine the sufficiency of geometric constraints on the two sets of point clouds in the target coordinate system.

[0047] The sufficiency of geometric constraints characterizes the magnitude of the effect of geometric constraints on points in a point cloud. Points with weak geometric constraints do not need to be associated with other point cloud pairs. Using the sufficiency of geometric constraints as one of the reference conditions for establishing associated point cloud pairs can avoid erroneous association of points with weak geometric constraints, thereby improving the efficiency of establishing associated point cloud pairs and enhancing the final point cloud registration accuracy and robustness.

[0048] Below, we will still use two sets of point clouds, namely point cloud P. j And point cloud P i The target coordinate system is the point cloud P. i Taking the coordinate system as an example, the implementation method for determining the sufficiency of geometric constraints will be further explained:

[0049] Semantic recognition was performed on the points contained in the two sets of point clouds. Based on the semantic recognition results, the points contained in the two sets of point clouds were divided into two categories: surface points and pole points. The semantic categories corresponding to surface points include signs, ground, walls, building surfaces, etc. The semantic categories corresponding to pole points include street lamp poles, utility poles, solid lane lines, dashed lane lines, curbs, etc.

[0050] For a link point, when the angle θ between the principal direction of the link point and the X-axis... x When the value is small, it can provide constraints in the X-axis direction, and the angle θ between the principal direction and the Y-axis. y When the angle is small, it can provide a constraint in the Y-axis direction; when the angle between the principal direction and the Z-axis is small, it can provide an angle θ in the Z-axis direction. z Constraints. The principal direction of a link-point pair is also the length direction or extension direction of the link-point.

[0051] Therefore, for the lever point, traverse the point cloud P.j Each pole point in ' and in the point cloud P i Search for the nearest pole point. If the distance to the nearest pole point is less than a distance threshold, calculate the angle between the pole point's principal direction and the X, Y, and Z axes, and set P... j The angle θ between the middle and the X-axis x The number of first pole points less than the angle threshold is determined as the constraint metric for pole points in the X-axis direction in both sets of point clouds; P j The angle θ between the middle and the Y-axis y The number of second pole points less than the angle threshold is determined as the constraint metric for pole points in the Y-axis direction in both sets of point clouds; P j The angle θ between the middle and the Z-axis z The number of third pole points less than the angle threshold is determined as the constraint metric of pole points in the Z-axis direction in both sets of point clouds.

[0052] For a point on a surface, the angle θ between the normal vector of the point and the X-axis is... x When the value is small, it can provide constraints in the X-axis direction, and the angle θ between the normal vector and the Y-axis. y When the angle is small, it can provide a constraint in the Y-axis direction; when the angle between the normal vector and the Z-axis is small, it can provide an angle θ in the Z-axis direction. z constraint.

[0053] Therefore, for a point cloud P, we need to traverse the point cloud. j Each facet in ' and in the point cloud P i Search for the nearest face in the array. If the distance to the nearest face is less than a distance threshold, calculate the angle between the normal vector of that face and the X, Y, and Z axes, and set P... j The angle θ between the middle and the X-axis x The number of first facet points less than the angle threshold is determined as the constraint metric for facet points in the X-axis direction in both point clouds; P j The angle θ between the middle and the Y-axis y The number of second face points less than the angle threshold is determined as the constraint metric for face points in the Y-axis direction in both point clouds; P j The angle θ between the middle and the Z-axis z The number of third face points less than the angle threshold is determined as the constraint metric of face points in the Z-axis direction in the two sets of point clouds.

[0054] The sufficiency of geometric constraints can be determined by the number of first, second, and third pole points, as well as the number of first, second, and third face points. Specifically, the constraint measures of pole points and face points in the X-axis direction, the Y-axis direction, and the Z-axis direction are added together to obtain the total constraint measure c of all pole points and face points in the two sets of point clouds. x c y c zAccording to the total constraint metric c x c y c z The minimum value in the formula can determine the sufficiency of the geometric constraints between the two sets of point clouds. For example, let c be the minimum value in the formula. x c y c z The ratio of the minimum value in the geometric constraint to the number of points in the cloud is used as the sufficiency level c of the geometric constraint. ij .

[0055] Step 101-4: Determine the correlation between the two sets of point clouds based on the degree of overlap and the sufficiency of geometric constraints.

[0056] In this embodiment of the disclosure, the correlation is calculated based on two dimensions: overlap and sufficiency of geometric constraints, in order to construct associated point cloud pairs. This can exclude point clouds with low correlation that are not suitable for optimizing the initial pose. On the one hand, this can reduce the computational load of subsequent relative pose calculation and initial pose optimization. On the other hand, after excluding point clouds with very low correlation, the subsequent point cloud registration results can be more accurate.

[0057] In one embodiment, the weighted result of overlap and sufficiency of geometric constraints is determined as the correlation between the two sets of point clouds, as expressed by the following formula:

[0058] s ij =ω o o ij +ω c c ij ;

[0059] Among them, s ij This represents the correlation between the i-th point cloud and the j-th point cloud, o ij c represents the overlap between the i-th point cloud and the j-th point cloud. ij ω represents the sufficiency of the geometric constraints between the i-th and j-th point clouds; o and ω c These are all weighting coefficients. The weighting coefficients can be set according to the actual situation, for example, all can be set to 0.5.

[0060] In one embodiment, before determining the correlation, the overlap and geometric constraint sufficiency are normalized. The correlation between the two sets of point clouds is then determined based on the normalized overlap and geometric constraint sufficiency, i.e., o in the above formula. ij and c ij The overlap and geometric constraint sufficiency are normalized.

[0061] In one embodiment, before determining the relevance, a score corresponding to the overlap and a score corresponding to the sufficiency of geometric constraints are calculated, and the weighted result of the two scores is determined as the relevance of the two sets of point clouds.

[0062] It should be noted that relevance is not limited to the weighted result of overlap and sufficiency of geometric constraints; it can also be the ratio of overlap to sufficiency of geometric constraints.

[0063] Step 102: Construct related point cloud pairs based on the correlation.

[0064] Each associated point cloud pair contains two sets of point clouds.

[0065] In one embodiment, two sets of point clouds with a correlation greater than a correlation threshold are identified as associated point cloud pairs. The correlation threshold can be set according to the actual situation.

[0066] Based on the correlation threshold, some pairwise point clouds were excluded. The excluded pairwise point clouds had very low correlation. On the one hand, this can reduce the amount of computation for subsequent relative pose calculation and initial pose optimization. On the other hand, after excluding pairwise point clouds with very low correlation, the subsequent point cloud registration results can be more accurate.

[0067] In one embodiment, before constructing the associated point cloud pair, semantic segmentation is performed on each point cloud to determine the semantic category of each point cloud, and two sets of point clouds with the same semantic category and a correlation greater than the correlation threshold are identified as associated point cloud pairs.

[0068] Understandably, registration is unnecessary when semantic categories differ. When establishing associated point cloud pairs, semantic categories are utilized, meaning that associations are performed within each semantic category, ensuring that the two points in an associated point cloud pair belong to the same semantic category. This reduces erroneous associations between points in the source and target point clouds, thereby improving the efficiency and accuracy of associated point cloud pair construction.

[0069] In one embodiment, a complete graph of multiple sets of point clouds is constructed (see...). Figure 3 In a complete graph, a node corresponds to a set of point clouds. The line connecting two sets of point clouds in the complete graph represents the correlation between the two sets of point clouds. The minimum spanning tree corresponding to the correlation is determined (see...). Figure 3 The two sets of point clouds connected by the line in the minimum spanning tree are identified as associated point cloud pairs.

[0070] In this embodiment, the minimum spanning tree algorithm is used to generate the minimum spanning tree corresponding to the complete graph. If there are N sets of point clouds, then the minimum spanning tree has N-1 edges. In this way, the number of registration edges can be reduced from O(N^2) to O(N^2). 2 The accuracy is reduced to O(N). Furthermore, the correlation between N-1 edges is relatively high, which is beneficial for obtaining better registration results.

[0071] In one embodiment, before constructing a complete graph of multiple point clouds, semantic segmentation is performed on each point cloud to determine its semantic category. Points in two frames with the same semantic category are connected by lines, while points in two frames with different semantic categories are not connected, thus constructing a complete graph. Therefore, combining semantic category and minimum spanning tree algorithm to determine associated point cloud pairs can further improve the efficiency and accuracy of associated point cloud pair construction, making subsequent point cloud registration results more accurate.

[0072] Step 103: For each associated point cloud pair, calculate the relative pose of the two sets of point clouds contained in the associated point cloud pair, and optimize the initial pose of the two sets of point clouds based on the relative pose.

[0073] In one embodiment, a semantic segmentation algorithm is used to calculate the relative poses of the two sets of point clouds contained in the associated point cloud pair. See [link to relevant documentation]. Figure 4 Step 104 specifically includes the following steps:

[0074] Step 103-1: Perform semantic segmentation on the two sets of point clouds respectively to obtain the semantic category of each set of point clouds.

[0075] This disclosure embodiment can employ any semantic segmentation algorithm from related technologies to perform semantic segmentation on point clouds. After semantic segmentation, the semantic category corresponding to each group of point clouds can be obtained. The semantic category can be represented by a semantic label.

[0076] For example, semantic tags can include: ground, fence, curb, vehicle, lane line, sign, pole, wall, other ground markings, etc.

[0077] Step 103-2: Calculate the relative pose of the two sets of point clouds based on the semantic category, the objective function corresponding to the rod-shaped object, and the objective function corresponding to the area-shaped object.

[0078] In this embodiment of the disclosure, when calculating the final relative pose, not only semantic categories are utilized, but constraints are also established through the objective functions corresponding to rod-shaped objects and surface-shaped objects. That is, the geometric information of the point cloud is combined, which greatly improves the accuracy and robustness of point cloud registration.

[0079] In one embodiment, the point clouds can be preprocessed before calculating the relative poses of the two sets of point clouds. For example... Figure 5 As shown, the preprocessing process includes S501-S504.

[0080] S501. Based on the straightness of each point in the point cloud whose semantic category belongs to the rod-shaped object, delete points whose straightness is less than the preset straightness threshold.

[0081] For points belonging to rod-shaped objects, if the straightness is less than the preset straightness threshold, it means that the point cannot be used as a point on the straight line corresponding to the rod-shaped object, and has little effect on geometric constraints, so the point can be deleted.

[0082] S502. Based on the flatness of each point in the point cloud whose semantic category belongs to the surface, delete points whose flatness is less than the preset flatness threshold.

[0083] For points belonging to planar objects, if the flatness of the point is less than the preset flatness threshold, it means that the point cannot be used as a point in the plane corresponding to the planar object, and has little effect on geometric constraints, so the point can be deleted.

[0084] S503. Perform density filtering on points in the point cloud whose semantic category belongs to ground markers.

[0085] Density filtering refers to obtaining the total number of points included in each type of ground marker in the point cloud, and determining whether the total number of points included in each type of ground marker is sufficient to represent the geometry corresponding to that type of ground marker. If not, the points included in that type of ground marker are deleted.

[0086] For example, if there are two points in a point cloud with the semantic category of arrow, obviously two points cannot represent an arrow shape, and therefore the points with the semantic category of arrow can be deleted from the point cloud.

[0087] S504. Perform ray tracing on each point in the point cloud whose semantic category belongs to the dynamic category, determine the static and dynamic states of each point, and delete points in motion.

[0088] It should be noted that the execution order of S501-S504 is not limited in the embodiments disclosed herein. Figure 2 For example, the execution is carried out in the order of S501-S504.

[0089] This method allows for the removal of points in the point cloud that have minimal geometric constraint effects, eliminating the need to calculate relative poses based on these points and preventing them from affecting the accuracy of the relative pose calculations. This improves the efficiency and accuracy of subsequent initial pose calculations and ultimately enhances the accuracy of the point cloud registration. Furthermore, removing points in motion eliminates the influence of dynamic objects on point cloud registration, improving the accuracy and robustness of point cloud registration in dynamic scenes.

[0090] The method for calculating relative pose is as follows:

[0091] Determine whether the geometric constraints represented by the associated point cloud pairs whose semantic category belongs to rod-shaped objects and the associated point cloud pairs whose semantic category belongs to area-shaped objects are sufficient.

[0092] If so, the relative pose is calculated based on the associated point cloud pairs whose semantic category belongs to rod-shaped objects and the associated point cloud pairs whose semantic category belongs to area-shaped objects.

[0093] If not, the relative pose is calculated based on the associated point cloud pairs whose semantic category belongs to rod-shaped objects, the associated point cloud pairs whose semantic category belongs to area-shaped objects, and the associated point cloud pairs whose semantic category belongs to ground markers.

[0094] For points that are rod-shaped, points whose principal direction is parallel to the x-axis can provide translation constraints in the y-axis and z-axis directions, points whose principal direction is parallel to the y-axis can provide translation constraints in the x-axis and z-axis directions, and points whose principal direction is parallel to the z-axis can provide translation constraints in the x-axis and y-axis directions.

[0095] Based on the above principles, for associated point cloud pairs belonging to rod-shaped objects, |o·X| can be used as the constraint contribution of each point with respect to the y-axis and z-axis directions, |o·Y| as the constraint contribution of each point with respect to the x-axis and z-axis directions, and |o·Z| as the constraint contribution of each point with respect to the x-axis and y-axis directions. Here, o represents the principal direction of the point. X, Y, and Z are the unit vectors in the x-axis direction, y-axis direction, and z-axis direction, respectively.

[0096] For points in the associated point cloud pairs belonging to the rod-shaped object, sum the |o·Y| and |o·Z| of all points to obtain the constraint contribution of the associated point cloud pairs belonging to the rod-shaped object in the x-axis direction; sum the |o·X| and |o·Z| of all points to obtain the constraint contribution of the associated point cloud pairs belonging to the rod-shaped object in the y-axis direction; sum the |o·X| and |o·Y| of all points to obtain the constraint contribution of the associated point cloud pairs belonging to the rod-shaped object in the z-axis direction.

[0097] By adding the constraint contributions in the x-axis direction, y-axis direction, and z-axis direction, the total geometric constraint contribution value of the associated point cloud pairs belonging to the rod-shaped object can be calculated.

[0098] For points belonging to a planar object, points whose normal vector is parallel to the x-axis can provide translation constraints in the x-axis direction, points whose normal vector is parallel to the y-axis can provide translation constraints in the y-axis direction, and points whose normal vector is parallel to the z-axis can provide translation constraints in the z-axis direction.

[0099] Based on the above principle, for a pair of associated point clouds belonging to a planar object, |n·X| can be used as the translation constraint of each point about the x-axis, |n·Y| as the translation constraint of each point about the y-axis, and |n·Z| as the translation constraint of each point about the y-axis, where n is the normal vector of the point.

[0100] For points in the associated point cloud pairs belonging to the planar object, sum |n·X| of all points to obtain the constraint contribution of the associated point cloud pairs belonging to the planar object in the x-axis direction; sum |n·Y| of all points to obtain the constraint contribution of the associated point cloud pairs belonging to the planar object in the y-axis direction; sum |n·Z| of all points to obtain the constraint contribution of the associated point cloud pairs belonging to the planar object in the z-axis direction.

[0101] Then, by adding the constraint contributions in the x-axis direction, y-axis direction, and z-axis direction, the total geometric constraint contribution value of the associated point cloud pairs belonging to the planar object can be calculated.

[0102] The total geometric constraint contribution values ​​of the associated point cloud pairs for rod-shaped objects and the associated point cloud pairs for area-shaped objects are summed to obtain the total geometric constraint contribution value for all associated point cloud pairs. If this total geometric constraint contribution value is greater than or equal to the constraint threshold, it can be determined that the geometric constraints represented by the associated point cloud pairs whose semantic category belongs to rod-shaped objects and the associated point cloud pairs whose semantic category belongs to area-shaped objects are sufficient; otherwise, the geometric constraints are determined to be insufficient.

[0103] Using the above method, we can accurately calculate whether the geometric constraints of the associated point cloud pairs belonging to rod-shaped objects and the associated point cloud pairs belonging to area-shaped objects are sufficient. When the geometric constraints are insufficient, we can further use the associated point cloud pairs of ground markers commonly found in urban scenes (such as pedestrian crossings, turning arrows, etc.) to calculate the relative pose. In this way, we can use the associated point cloud pairs of ground markers to provide additional geometric constraints, improve the accuracy of the calculated relative pose, and thus improve the robustness of point cloud registration.

[0104] Given sufficient geometric constraints, the relative pose can be calculated based on the objective functions corresponding to the rod-shaped object and the surface-shaped object.

[0105] The objective function for the rod-shaped object is the objective function from the point to the line, and the specific formula is as follows:

[0106] d pole (i)=||o i ×(q i -Rp i -t)||;

[0107] Among them, o i p represents the main direction of the point cloud. i and q i For a pair of correlated point clouds, p i For a point in the source point cloud, q i Let R be a point in the target point cloud, ∈ SO(3), where R is the rotation matrix and SO(3) is the three-dimensional rotation group. t is the translation vector. It is a three-dimensional Euclidean space.

[0108] The objective function for a surface is the objective function from point to surface, and the specific formula is as follows:

[0109] d plane (i)=|n i ·(q i -Rp i -t)|

[0110] Where, n i Let be the normal vector of the point.

[0111] Accordingly, the relative pose is:

[0112]

[0113] Among them, R * For the calculated rotation matrix, t * The calculated translation matrix is ​​argmin, where N is the set of all independent variables that minimize the function. pole N represents the number of associated point cloud pairs for rod-shaped objects. plane This represents the number of associated point cloud pairs for the area.

[0114] When the geometric constraints are insufficient, the relative pose can be calculated based on the objective functions corresponding to the rod-shaped objects, the surface-shaped objects, and the ground markers.

[0115] The objective function for the ground markers is the objective function for the distance between marker points, and the specific formula is as follows:

[0116] d point (i)=||q i -Rp i -t||;

[0117] Accordingly, the relative pose is:

[0118]

[0119] N point The number of associated point cloud pairs for ground markers.

[0120] Using this method, when geometric constraints are insufficient, ground markers can be used to assist in point cloud registration, which can avoid the problem of solution degradation caused by insufficient geometric constraints and further improve the accuracy of point cloud registration.

[0121] In the above embodiments, the method for determining whether the relative pose has converged is as follows:

[0122] If the difference between the calculated relative pose and the rotation angle of the rotation matrix in the current relative pose is less than the third angle threshold, and the difference between the translation distance of the translation matrix and the translation distance threshold is less than the translation distance threshold, then the calculated relative pose is determined to be converged; otherwise, the calculated relative pose is determined to be non-converged.

[0123] If the relative poses obtained from the two calculations are close, it can be determined that the calculated relative poses have converged. This can improve computational efficiency while ensuring accuracy, and can also obtain more accurate relative poses, thereby improving the accuracy of point cloud registration.

[0124] Based on the above embodiments, the final relative pose between the source point cloud and the target point cloud is calculated based on the associated point cloud pairs and geometric information in the source point cloud and the target point cloud. Specifically, this can be achieved through the following steps:

[0125] The relative pose is calculated based on the associated point cloud pairs in the source and target point clouds. The process then determines whether the calculated relative pose has converged. If converged, the calculated relative pose is used as the final relative pose between the source and target point clouds. If not converged, the calculated relative pose is used as the current relative pose, and the process returns to the step of transforming the points in the source point cloud to the target point cloud coordinate system based on the current relative pose, until the final relative pose is obtained.

[0126] Using this method, the relative pose of the source point cloud and the target point cloud can be calculated using the associated point cloud pair. If the relative pose does not converge, the associated point cloud pair can be reconstructed based on the relative pose obtained in this calculation. By iterating, the accurate relative pose between the source point cloud and the target point cloud can be calculated step by step, which can improve the accuracy of the final calculated relative pose.

[0127] Step 103-3: Optimize the initial pose of the two sets of point clouds contained in the associated point cloud pair based on the relative pose.

[0128] Relative pose includes rotation and translation matrices, which can be used to transform each point in the point cloud, thereby optimizing the initial pose of the point cloud.

[0129] Step 104: Perform multi-view point cloud registration on multiple sets of point clouds based on the optimized initial pose.

[0130] In this embodiment of the disclosure, the initial pose of the point cloud is optimized based on the relative pose of the constructed associated point cloud pairs, so as to provide a high-precision initial pose for subsequent multi-view point registration, thereby reducing the number of multi-view point registration iterations and improving the accuracy of multi-view point registration.

[0131] Multi-view point cloud registration is performed on the multiple sets of point clouds based on the optimized initial pose.

[0132] The embodiments disclosed herein can employ any of the multi-view point cloud registration algorithms in the related technologies for multi-view point cloud registration.

[0133] The following provides an implementation method for multi-view point cloud registration.

[0134] The optimization variable for point-to-point registration is the relative pose from the source point cloud to the target point cloud, while the optimization variable for multi-view simultaneous registration is the pose of all point clouds. In this embodiment, point clouds of different semantic categories are voxelized separately. Assume that a voxel after voxelization contains N points p. i Let i = 1, 2, ..., r, and point p. i Corresponding frame s i , where s i =1,2,…,M, one frame s i For one or more point clouds, frames s i The corresponding optimized pose is Assume the covariance matrix of N points within this voxel is C, and its corresponding eigenvalues ​​are λ. k Let k = 1, 2, 3 be the k-th largest eigenvalue. For face voxels, minimizing the eigenvalue λ1 can be used as the optimization objective. By minimizing eigenvalue λ1, the thickness of the face is minimized, thus achieving multi-view point cloud registration. For line voxels, minimizing the eigenvalue λ2 can be used as the optimization objective. By minimizing eigenvalue λ2, the width of the line is minimized, thus achieving multi-view point cloud registration. The optimization formula for multi-view point cloud registration can be, but is not limited to, expressed as follows:

[0135]

[0136]

[0137]

[0138]

[0139]

[0140]

[0141] Among them, T * The optimization objective for multi-view point cloud registration is represented by T; T represents the optimized pose of the point clouds across all frames; u k λ k The corresponding eigenvectors; R j P represents the rotation matrix of the j-th frame; oi Represents the original coordinates in the coordinate system of the cloud data acquisition device (such as a laser scanning radar); This represents a 3x6 vector of zeros.

[0142] The optimization formula for multi-view point cloud registration can be solved using the LM algorithm.

[0143] Corresponding to the aforementioned point cloud registration method embodiments, this disclosure also provides embodiments of point cloud registration devices.

[0144] Figure 6 A schematic diagram of a high-precision map point cloud registration device provided for an exemplary embodiment of this disclosure, the point cloud registration device comprising:

[0145] The determination module 61 is used to acquire multiple sets of point clouds to be matched and determine the correlation between pairs of point clouds in the multiple sets of point clouds.

[0146] The construction module 62 is used to construct associated point cloud pairs based on the correlation; wherein each associated point cloud pair contains two sets of point clouds;

[0147] The optimization module 63 is used to calculate the relative pose of the two sets of point clouds contained in each associated point cloud pair, and optimize the initial pose of the two sets of point clouds based on the relative pose.

[0148] The registration module 64 is used to perform multi-view point cloud registration on the multiple sets of point clouds based on the optimized initial pose.

[0149] Optionally, the determining module includes:

[0150] The transformation unit is used to unify the initial poses of two sets of point clouds into the target coordinate system;

[0151] An overlap determination unit is used to determine the overlap between two sets of point clouds in the target coordinate system.

[0152] The constraint determination unit is used to determine the sufficiency of geometric constraints on the two sets of point clouds in the target coordinate system.

[0153] A correlation determination unit is used to determine the correlation degree based on the overlap degree and the sufficiency of the geometric constraints.

[0154] Optionally, the overlap determination unit is specifically used for:

[0155] Determine the number of target point pairs contained in the two sets of point clouds; wherein, the target point pair is two points in the target coordinate system that belong to the two sets of point clouds respectively and whose distance is less than a first distance threshold;

[0156] The overlap is determined based on the quantity; wherein the overlap is positively correlated with the quantity.

[0157] Optionally, the constraint determination unit is specifically used for:

[0158] Determine the number of first-axis rods providing geometric constraints in the X-axis direction, the number of second-axis rods providing geometric constraints in the Y-axis direction, and the number of third-axis rods providing geometric constraints in the Z-axis direction in the two sets of point clouds.

[0159] Determine the number of first facets providing geometric constraints in the X-axis direction, the number of second facets providing geometric constraints in the Y-axis direction, and the number of third facets providing geometric constraints in the Z-axis direction in the two sets of point clouds.

[0160] The sufficiency of the geometric constraints is determined based on the number of the first link point, the second link point, the third link point, the first face point, the second face point, and the third face point.

[0161] Optionally, the building module is specifically used for:

[0162] Construct a complete graph of the multiple sets of point clouds, where one node of the complete graph corresponds to one set of point clouds, and the connection between two sets of point clouds in the complete graph represents the correlation between the two sets of point clouds.

[0163] Determine the minimum spanning tree corresponding to the relevance;

[0164] The two sets of point clouds connected by a line in the minimum spanning tree are defined as the associated point cloud pair.

[0165] Optionally, when calculating the relative pose of the two sets of point clouds contained in the associated point cloud pair, the optimization module is specifically used for:

[0166] Semantic segmentation was performed on the two sets of point clouds to obtain the semantic category of each set of point clouds;

[0167] The relative poses of the two sets of point clouds are determined based on the semantic category, the objective function corresponding to the rod-shaped object, and the objective function corresponding to the area-shaped object.

[0168] Optionally, the distance between two sets of point clouds in the pairwise point clouds is less than a second distance threshold;

[0169] And / or, the semantic categories of two pairs of point clouds are the same.

[0170] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this disclosure according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0171] The technical solutions disclosed herein involve the collection, storage, use, processing, transmission, provision, and disclosure of point cloud data, all of which comply with relevant laws and regulations and do not violate public order and good morals.

[0172] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0173] Figure 7 A schematic block diagram of an example electronic device 700 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0174] like Figure 7 As shown, device 700 includes a computing unit 701, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 702 or a computer program loaded from storage unit 708 into random access memory (RAM) 703. RAM 703 may also store various programs and data required for the operation of device 700. The computing unit 701, ROM 702, and RAM 703 are interconnected via bus 704. Input / output (I / O) interface 705 is also connected to bus 704.

[0175] Multiple components in device 700 are connected to I / O interface 705, including: input unit 706, such as keyboard, mouse, etc.; output unit 707, such as various types of monitors, speakers, etc.; storage unit 708, such as disk, optical disk, etc.; and communication unit 709, such as network card, modem, wireless transceiver, etc. Communication unit 709 allows device 700 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0176] The computing unit 701 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 701 performs the various methods and processes described above, such as the point cloud registration method. For example, in some embodiments, the point cloud registration method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 708. In some embodiments, part or all of the computer program may be loaded and / or installed on device 700 via ROM 702 and / or communication unit 709. When the computer program is loaded into RAM 703 and executed by computing unit 701, one or more steps of the point cloud registration method described above may be performed. Alternatively, in other embodiments, the computing unit 701 may be configured to perform the point cloud registration method by any other suitable means (e.g., by means of firmware).

[0177] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0178] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0179] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0180] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0181] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0182] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0183] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0184] This disclosure also provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause the computer to execute the point cloud registration method provided according to any of the above embodiments.

[0185] This disclosure also provides a computer program product, including a computer program that, when executed by a processor, implements the point cloud registration method provided according to any of the above embodiments.

[0186] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A method for high-precision map point cloud registration, comprising: Obtain multiple sets of point clouds to be matched, and determine the correlation between pairs of point clouds in the multiple sets of point clouds; Based on the correlation, associated point cloud pairs are constructed; wherein each associated point cloud pair contains two sets of point clouds; For each associated point cloud pair, calculate the relative pose of the two sets of point clouds contained in the associated point cloud pair, and optimize the initial pose of the two sets of point clouds based on the relative pose. Based on the optimized initial pose, multi-view point cloud registration is performed on the multiple sets of point clouds. Determining the correlation between pairs of point clouds in the multiple point clouds includes: Unify the initial poses of the two point clouds into the target coordinate system; Determine the overlap between the two sets of point clouds in the target coordinate system; Determine the sufficiency of the geometric constraints of the two sets of point clouds in the target coordinate system; The correlation is determined based on the degree of overlap and the sufficiency of the geometric constraints.

2. The high-precision map point cloud registration method according to claim 1, wherein, Determining the overlap between the two sets of point clouds in the target coordinate system includes: Determine the number of target point pairs contained in the two sets of point clouds; wherein, the target point pair is two points in the target coordinate system that belong to the two sets of point clouds respectively and whose distance is less than a first distance threshold; The overlap is determined based on the quantity; wherein the overlap is positively correlated with the quantity.

3. The high-precision map point cloud registration method according to claim 1, wherein, Determining the sufficiency of geometric constraints on the two sets of point clouds in the target coordinate system includes: Determine the number of first-axis rods providing geometric constraints in the X-axis direction, the number of second-axis rods providing geometric constraints in the Y-axis direction, and the number of third-axis rods providing geometric constraints in the Z-axis direction in the two sets of point clouds. Determine the number of first facets providing geometric constraints in the X-axis direction, the number of second facets providing geometric constraints in the Y-axis direction, and the number of third facets providing geometric constraints in the Z-axis direction in the two sets of point clouds. The sufficiency of the geometric constraints is determined based on the number of the first link point, the number of the second link point, the number of the third link point, the number of the first face point, the number of the second face point, and the number of the third face point.

4. The high-precision map point cloud registration method according to claim 1, wherein, The step of constructing associated point cloud pairs based on the correlation includes: Construct a complete graph of the multiple sets of point clouds, where one node of the complete graph corresponds to one set of point clouds, and the connection between two sets of point clouds in the complete graph represents the correlation between the two sets of point clouds. Determine the minimum spanning tree corresponding to the relevance based on the complete graph; The two sets of point clouds connected by a line in the minimum spanning tree are defined as the associated point cloud pair.

5. The high-precision map point cloud registration method according to any one of claims 1-4, wherein, The calculation of the relative pose of the two sets of point clouds contained in the associated point cloud pair includes: Semantic segmentation was performed on the two sets of point clouds to obtain the semantic category of each set of point clouds; The relative poses of the two sets of point clouds are determined based on the semantic category, the objective function corresponding to the rod-shaped object, and the objective function corresponding to the area-shaped object.

6. The high-precision map point cloud registration method according to claim 5, wherein the distance between the two sets of point clouds in the pairwise point clouds is less than a second distance threshold; And / or, the semantic categories of two pairs of point clouds are the same.

7. A high-precision map point cloud registration device, comprising: The determination module is used to acquire multiple sets of point clouds to be matched and determine the correlation between pairs of point clouds in the multiple sets of point clouds. A construction module is used to construct associated point cloud pairs based on the correlation; wherein each associated point cloud pair contains two sets of point clouds; An optimization module is used to calculate the relative pose of the two sets of point clouds contained in each associated point cloud pair, and optimize the initial pose of the two sets of point clouds based on the relative pose. The registration module is used to perform multi-view point cloud registration on the multiple sets of point clouds based on the optimized initial pose. The determining module includes: The transformation unit is used to unify the initial poses of two sets of point clouds into the target coordinate system; An overlap determination unit is used to determine the overlap between two sets of point clouds in the target coordinate system. The constraint determination unit is used to determine the sufficiency of geometric constraints on the two sets of point clouds in the target coordinate system. A correlation determination unit is used to determine the correlation degree based on the overlap degree and the sufficiency of the geometric constraints.

8. The high-precision map point cloud registration device according to claim 7, wherein, The overlap determination unit is specifically used for: Determine the number of target point pairs contained in the two sets of point clouds; wherein, the target point pair is two points in the target coordinate system that belong to the two sets of point clouds respectively and whose distance is less than a first distance threshold; The overlap is determined based on the quantity; wherein the overlap is positively correlated with the quantity.

9. The high-precision map point cloud registration device according to claim 7, wherein, The constraint determination unit is specifically used for: Determine the number of first-axis rods providing geometric constraints in the X-axis direction, the number of second-axis rods providing geometric constraints in the Y-axis direction, and the number of third-axis rods providing geometric constraints in the Z-axis direction in the two sets of point clouds. Determine the number of first facets providing geometric constraints in the X-axis direction, the number of second facets providing geometric constraints in the Y-axis direction, and the number of third facets providing geometric constraints in the Z-axis direction in the two sets of point clouds. The sufficiency of the geometric constraints is determined based on the number of the first link point, the second link point, the third link point, the first face point, the second face point, and the third face point.

10. The high-precision map point cloud registration device according to claim 7, wherein, The building module is specifically used for: Construct a complete graph of the multiple sets of point clouds, where one node of the complete graph corresponds to one set of point clouds, and the connection between two sets of point clouds in the complete graph represents the correlation between the two sets of point clouds. Determine the minimum spanning tree corresponding to the relevance based on the complete graph; The two sets of point clouds connected by a line in the minimum spanning tree are defined as the associated point cloud pair.

11. The high-precision map point cloud registration device according to any one of claims 7-10, wherein, When calculating the relative pose of the two sets of point clouds contained in the associated point cloud pair, the optimization module is specifically used for: Semantic segmentation was performed on the two sets of point clouds to obtain the semantic category of each set of point clouds; The relative poses of the two sets of point clouds are determined based on the semantic category, the objective function corresponding to the rod-shaped object, and the objective function corresponding to the area-shaped object.

12. The high-precision map point cloud registration device according to claim 11, wherein the distance between two sets of point clouds in the pairwise point clouds is less than a second distance threshold; And / or, the semantic categories of two pairs of point clouds are the same.

13. An electronic device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the point cloud registration method according to any one of claims 1-6.

14. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to execute the point cloud registration method according to any one of claims 1-6.

15. A computer program product comprising a computer program that, when executed by a processor, implements the point cloud registration method according to any one of claims 1-6.

Images