Method, device, electronic device and storage medium for high-precision map point cloud registration

Abstract

The present disclosure provides a method, device, electronic device and storage medium for high-precision map point cloud registration, which relates to the field of data processing technology, especially to the field of autonomous driving and high-precision map technology. The specific implementation scheme is as follows: semantic segmentation is performed on the source point cloud and the target point cloud respectively to obtain the semantic category of each point in the source point cloud and the target point cloud; the initial relative pose is used as the current relative pose, and the points in the source point cloud are transformed into the target point cloud coordinate system based on the current relative pose; the initial relative pose is the relative pose between the pre-measured source point cloud and the target point cloud; based on the position of each point in the source point cloud in the target point cloud coordinate system, the nearest neighbor point with the same semantic category as the point in the target point cloud is searched, and the point and the nearest neighbor point corresponding to the point are regarded as an associated point pair; the final relative pose between the source point cloud and the target point cloud is calculated based on the associated point pairs and geometric information in the source point cloud and the target point cloud. The registration error can be reduced.

Description

Technical Field

[0001] The present disclosure relates to the field of data processing technology, and in particular to the field of autonomous driving and high-precision map technology. Background Art

[0002] High-precision maps, also known as high-accuracy maps, are used by autonomous vehicles. These maps contain precise vehicle location information and rich road element data, helping vehicles predict complex road conditions such as slope, curvature, and heading, helping them better avoid potential risks. Point cloud registration algorithms can be used in the production of high-precision maps. Point cloud registration involves finding the relative pose that accurately represents the relationship between two point clouds. Summary of the Invention

[0003] The present disclosure provides a method, device, electronic device, and storage medium for high-precision map point cloud registration.

[0004] According to a first aspect of the present disclosure, a method for high-precision map point cloud registration is provided, comprising:

[0005] Perform semantic segmentation on the source point cloud and the target point cloud respectively to obtain the semantic category of each point in the source point cloud and the target point cloud;

[0006] Using an initial relative pose as a current relative pose, and transforming points in the source point cloud to a target point cloud coordinate system based on the current relative pose; the initial relative pose is a pre-measured relative pose between the source point cloud and the target point cloud;

[0007] Based on the position of each point in the source point cloud in the target point cloud coordinate system, searching for the nearest neighbor point in the target point cloud with the same semantic category as the point, and treating the point and its corresponding nearest neighbor point as an associated point pair;

[0008] A final relative pose between the source point cloud and the target point cloud is calculated based on associated point pairs and geometric information in the source point cloud and the target point cloud.

[0009] According to a second aspect of the present disclosure, a device for high-precision map point cloud registration is provided, comprising:

[0010] The semantic segmentation module is used to perform semantic segmentation on the source point cloud and the target point cloud respectively to obtain the semantic category of each point in the source point cloud and the target point cloud;

[0011] a coordinate transformation module, configured to transform the points in the source point cloud into a target point cloud coordinate system based on an initial relative pose as a current relative pose; the initial relative pose being a pre-measured relative pose between the source point cloud and the target point cloud;

[0012] A point cloud association module searches for a nearest neighbor point in the target point cloud with the same semantic category as each point in the source point cloud based on the position of the point in the target point cloud coordinate system, and treats the point and its corresponding nearest neighbor point as an association point pair;

[0013] A calculation module is used to calculate the final relative pose between the source point cloud and the target point cloud based on the associated point pairs and set information in the source point cloud and the target point cloud.

[0014] According to a third aspect of the present disclosure, there is provided an electronic device, including:

[0015] at least one processor; and

[0016] a memory communicatively connected to the at least one processor; wherein,

[0017] The memory stores instructions that can be executed by the at least one processor. The instructions are executed by the at least one processor to enable the at least one processor to perform the method according to the first aspect.

[0018] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause the computer to execute the method described in the first aspect.

[0019] According to a fifth aspect of the present disclosure, a computer program product is provided, comprising a computer program, wherein the computer program implements the method described in the first aspect when executed by a processor.

[0020] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The accompanying drawings are provided to facilitate a better understanding of the present invention and do not constitute a limitation of the present disclosure.

[0022] Figure 1 This is a flow chart of a first method for high-precision map point cloud registration provided according to an embodiment of the present disclosure;

[0023] Figure 2 This is a flow chart of a second method for high-precision map point cloud registration provided according to an embodiment of the present disclosure;

[0024] Figure 3 This is an exemplary flow chart of a high-precision map point cloud registration method provided according to an embodiment of the present disclosure;

[0025] Figure 4This is a schematic structural diagram of a high-precision map point cloud registration device provided according to an embodiment of the present disclosure;

[0026] Figure 5 It is a block diagram of an electronic device used to implement the method of high-precision map point cloud registration in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0027] The following description of exemplary embodiments of the present disclosure is made in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding. These details should be considered as merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0028] Currently, point cloud registration algorithms play an important role in the production of high-precision maps. Commonly used point cloud registration algorithms include the Iterative Closest Point (ICP) algorithm and its variants, and the Normal Distribution Transform (NDT) algorithm and its variants. These algorithms mainly use the geometric information of the point cloud for point cloud registration, and the point cloud registration error is large.

[0029] To solve the above problems, the present disclosure provides a method for high-precision map point cloud registration. Figure 1 As shown, the method includes:

[0030] S101. Perform semantic segmentation on the source point cloud and the target point cloud respectively to obtain the semantic category of each point in the source point cloud and the target point cloud.

[0031] The embodiments of the present disclosure can use any semantic segmentation algorithm in related technologies to perform semantic segmentation on the source point cloud and the target point cloud. After semantic segmentation, the semantic category corresponding to each point in the source point cloud and the target point cloud can be obtained. The semantic category can be represented by a semantic label.

[0032] For example, semantic labels may include: ground, fence, curb, vehicle, lane marking, sign, pole, wall, other ground markings, etc.

[0033] S102: Taking the initial relative pose as the current relative pose, and transforming the points in the source point cloud to the target point cloud coordinate system based on the current relative pose.

[0034] The initial relative pose is the pre-measured relative pose between the source point cloud and the target point cloud.

[0035] When collecting source point clouds and target point clouds, the Global Positioning System (GPS) information of the lidar when collecting the source point cloud and the GPS information of the lidar when collecting the target point cloud can be recorded, and the initial relative pose can be calculated based on these two GPS information.

[0036] The current relative pose includes a rotation matrix and a translation matrix, and each point in the current point cloud can be transformed based on the rotation matrix and the translation matrix, thereby mapping each point in the source point cloud to the target point cloud coordinate system.

[0037] S103 , based on the position of each point in the source point cloud in the target point cloud coordinate system, searching for the nearest neighbor point in the target point cloud that has the same semantic category as the point, and treating the point and its corresponding nearest neighbor point as an associated point pair.

[0038] After mapping each point in the source point cloud to the target point cloud coordinate system, the nearest neighbor point of each semantic category in the source point cloud can be searched for in the target point cloud. Specifically, for a point in the source point cloud, the target point cloud can be searched for a point with the same semantic category as the point, and the point closest to the point in the target point cloud coordinate system can be used as the nearest neighbor point of the point. It can be understood that since there is no need to search for points of different semantic categories for the point, the search volume can be reduced, and the association efficiency can be improved.

[0039] For example, consider point A in the source point cloud, whose semantic category is streetlight pole. The target point cloud includes 1,000 points with the semantic category streetlight pole. If point B is closest to point A in the target point cloud coordinate system, point B can be determined to be the nearest neighbor of point A. Points A and B can be considered a connected point pair.

[0040] S104 : Calculate the final relative pose between the source point cloud and the target point cloud based on the associated point pairs and geometric information in the source point cloud and the target point cloud.

[0041] Using this method, semantic categories are used when establishing associated point pairs, that is, associations are performed within each semantic category so that the two points in an associated point pair belong to the same semantic category. This can reduce the misassociation of points in the source point cloud and the target point cloud. When calculating the final relative pose, not only the geometric information in the source point cloud and the target point cloud is used, but also the associated point pairs with higher accuracy established by the semantic categories mentioned above are used, which can improve the accuracy and robustness of point cloud registration.

[0042] In some embodiments of the present disclosure, after performing semantic segmentation on the source point cloud and the target point cloud respectively in the above S101 to obtain the semantic category of each point in the source point cloud and the target point cloud, the source point cloud and the target point cloud may be preprocessed, such as Figure 2 As shown, the pre-processing process includes S201-S204.

[0043] S201 , based on the straightness of each point in the source point cloud and the target point cloud whose semantic category belongs to the rod-shaped object, delete the points whose straightness is less than a preset straightness threshold.

[0044] Among them, for a point belonging to a rod-shaped object, if the straightness is less than the preset straightness threshold, it means that the point cannot be used as a point on the straight line corresponding to the rod-shaped object and has little effect on the geometric constraint, so the point can be deleted.

[0045] S202 , based on the planarity of each point in the source point cloud and the target point cloud whose semantic category belongs to planar objects, delete points whose planarity is less than a preset planarity threshold.

[0046] Among them, for a point belonging to a planar object, if the flatness of the point is less than the preset flatness threshold, it means that the point cannot be used as a point in the plane corresponding to the planar object and has little effect on the geometric constraint, so the point can be deleted.

[0047] S203 : Density filtering is performed on points in the source point cloud and the target point cloud whose semantic categories belong to ground markers.

[0048] Among them, density filtering refers to obtaining the total number of points included in each ground marker in the point cloud, and whether the total number of points included in each ground marker is sufficient to represent the geometric figure corresponding to the ground marker. If not, the points included in the ground marker are deleted.

[0049] For example, if there are two points with the semantic category of arrow in the source point cloud, it is obvious that two points cannot represent an arrow shape, and the points with the semantic category of arrow can be deleted from the source point cloud.

[0050] S204 , performing ray tracing on each point in the source point cloud and the target point cloud whose semantic category belongs to the dynamic category, determining the dynamic or static state of each point, and deleting points in a moving state.

[0051] It should be noted that the embodiment of the present disclosure does not limit the execution order between S201-S204. Figure 2 Here, the execution in the order of S201 to S204 is taken as an example.

[0052] This method can remove points in the source and target point clouds that have little effect on geometric constraints, eliminating the need to establish point pairs for these points and preventing subsequent misassociations between these points. This improves the efficiency of establishing subsequent point pairs and the accuracy of the final point cloud registration. Furthermore, by removing points in motion, the influence of dynamic objects on point cloud registration can be eliminated, improving the accuracy and robustness of point cloud registration in dynamic scenes.

[0053] In some embodiments of the present disclosure, a pole-shaped object may correspond to multiple semantic categories, such as a streetlight pole, a utility pole, a solid lane line, a dashed lane line, a curb, and other semantic categories. The multiple semantic categories corresponding to pole-shaped objects can be divided into two categories: a first preset semantic category and a second preset semantic category.

[0054] The first preset semantic category includes semantic categories corresponding to pole-shaped objects that can be fitted into straight lines, such as streetlight poles, electric poles, and dashed lane segments.

[0055] The second preset semantic category includes semantic categories corresponding to rod-shaped objects that cannot be modeled as straight lines, such as lane lines and curbs. As you can understand, lane lines curve along the curvature of the road and therefore cannot be modeled as straight lines.

[0056] The embodiment of the present disclosure can preprocess the points of the first preset semantic category and the second preset semantic category respectively, that is, based on the straightness of each point in the source point cloud and the target point cloud whose semantic category belongs to the rod-shaped object, the points whose straightness is less than the preset straightness threshold are deleted, which can specifically include the preprocessing of the points of the first preset semantic category and the preprocessing of the points of the second preset semantic category.

[0057] The preprocessing of points of the first preset semantic category includes:

[0058] Cluster fitting is performed on the points of the first preset semantic category in the source point cloud and the target point cloud respectively, and the first point sets constituting the straight lines of the rod-shaped object in the source point cloud and the target point cloud are obtained respectively, and the points in the first point sets whose straightness is less than the preset straightness threshold are deleted.

[0059] Specifically, points of the first preset semantic category in the source point cloud and the target point cloud may be clustered respectively to obtain at least one rod instance in the source point cloud and at least one rod instance in the target point cloud.

[0060] Among them, points belonging to the first preset semantic category in the source point cloud can be screened out, and for the screened points, Euclidean distance clustering is performed on points with the same semantic label, and each type of point obtained by clustering is regarded as a pole instance.

[0061] Similarly, points belonging to the first preset semantic category in the target point cloud can also be screened out, and for the screened points, Euclidean distance clustering is performed on points with the same semantic label, and each type of point obtained is regarded as a pole instance.

[0062] For example, if both the source point cloud and the target point cloud include multiple points with the semantic label of streetlight poles, the pole instances of the streetlight poles in the source point cloud and the pole instances of the streetlight poles in the target point cloud can be obtained by clustering.

[0063] Then, for each rod instance, iterative straight line fitting can be performed on the points included in the rod instance until the number of points in the rod instance that are not fitted into straight lines is less than a first threshold, and the points included in each straight line obtained by fitting are respectively regarded as a first point set.

[0064] In the embodiment of the present disclosure, a Random Sample Consensus (RANSAC) algorithm may be used for iterative straight line fitting. The first threshold is a preset empirical value. If the number of points is less than the first threshold, it means that these points are insufficient to fit a straight line.

[0065] For example, if a large electric pole is composed of four closely spaced small electric poles, Euclidean distance clustering may yield a pole instance corresponding to the large pole. Suppose the pole instance contains 1000 points. RANSAC can then be used to fit a straight line from these 1000 points. Suppose the straight line contains 250 points, and these 250 points are then removed from the pole instance.

[0066] Then, a straight line is fitted for the remaining 750 points using the RANSAC algorithm. Assuming that the straight line includes 260 points, these 260 points are deleted from the rod instance.

[0067] Then, a straight line is fitted for the remaining 490 points using the RANSAC algorithm. Assuming that the straight line includes 240 points, these 240 points are deleted from the rod instance.

[0068] Then, use the RANSAC algorithm to fit a straight line to the remaining 250 points. Assuming this line includes 240 points, remove these 240 points from the rod instance, leaving only 10 points. Four straight lines have been fitted. Since 10 is less than the first threshold, the remaining 10 points can be removed.

[0069] Through the above-mentioned iterative straight-line fitting process, even if multiple rod-shaped objects are mistakenly clustered into one rod instance, the straight line corresponding to each rod-shaped object can be separated by iterative straight-line fitting, which can avoid fitting multiple closely spaced rod-shaped objects into one straight line, thereby accurately deleting points that do not contribute to the geometric constraints, and avoiding the mistaken deletion of points that contribute to the geometric constraints, thereby improving the accuracy of subsequent point cloud registration.

[0070] Each point in the fitted line satisfies the condition that the distance from the point to the line is less than a distance threshold. For example, the distance threshold may be 0.1 meter. If the fitted line includes 250 points, then the distance from each of these 250 points to the fitted line is less than 0.1 meter. The principal direction of the fitted line can be used as the principal direction of each point within the rod-shaped object corresponding to the line.

[0071] Based on each straight line fitted above, the standard deviation σ of the points included in the straight line is calculated according to the distance from the point corresponding to the straight line to the straight line, and the straightness of the fitted line is calculated using exp{-Cσ}.

[0072] Among them, exp stands for exponential, which is an exponential function in higher mathematics with the natural constant e as the base, and C is the preset parameter.

[0073] In the embodiment of the present disclosure, after the straightness of a line is calculated using the above formula, the straightness of the line is used as the straightness of each point corresponding to the line. The straightness of each point can be used as a feature quality score, and the points with a feature quality score less than a preset straightness threshold (s) are deleted. pole ) point.

[0074] Since the straightness of all points corresponding to a straight line is the same as the straightness of the straight line, if the straightness of the straight line is less than the preset straightness threshold (s pole ), all points included in the line will be deleted.

[0075] For example, the preset straightness threshold (s pole ) is 1. If the semantic category of the points corresponding to the straight line is a telephone pole, and the straightness of the line is 0.7, it indicates that the pole may have a curved branch, and the points corresponding to the curved branch are also fitted into a straight line. In this case, all the points corresponding to the straight line can be deleted.

[0076] The preprocessing of points of the second preset semantic category includes:

[0077] For each point of the second preset semantic category in the source point cloud and the target point cloud, the straightness of each point is calculated based on the covariance matrix of the points in the preset neighborhood range of the point, and points with straightness less than the preset straightness threshold are deleted.

[0078] In the embodiment of the present disclosure, each point of the second preset semantic category in the source point cloud and the target point cloud can be traversed, and the main direction of each point can be approximately calculated using the distribution of points within a preset neighborhood range of each point.

[0079] Specifically, for each point, first search for points within the preset neighborhood of the point, then calculate the covariance matrix of the searched points, perform eigenvalue decomposition based on the covariance matrix, and use the eigenvector corresponding to the maximum eigenvalue λ3 obtained by eigenvalue decomposition as the main direction of the point, and use the straightness (λ3-λ2) / λ3 as the feature quality score of the point, and delete the points with a feature quality score less than the preset straightness threshold (s pole ) point.

[0080] By preprocessing the points of the first preset semantic category and the second preset semantic category, it is possible to avoid the points that do not actually belong to the straight line and the points that do not contribute much to the geometric constraints from affecting the point cloud registration results, thereby improving the point cloud registration accuracy.

[0081] Similar to pole-shaped objects, surface-shaped objects can include multiple semantic categories, such as signs, ground, walls, building surfaces, etc.

[0082] The various semantic categories corresponding to planar objects can be divided into two categories: a third preset semantic category and a fourth preset semantic category.

[0083] The third preset semantic category includes semantic categories corresponding to planar objects, which can be fitted into a plane, such as signboards.

[0084] The fourth preset semantic category includes semantic categories corresponding to planar objects that cannot be fitted into a plane, such as ground, wall, building plane, etc.

[0085] The embodiment of the present disclosure can preprocess the points of the third preset semantic category and the fourth preset semantic category respectively, that is, the above S202, based on the flatness of each point whose semantic category belongs to the planar object in the source point cloud and the target point cloud, delete the points whose flatness is less than the preset flatness threshold. Specifically, it can include the preprocessing of the points of the third preset semantic category and the preprocessing of the points of the fourth preset semantic category.

[0086] The preprocessing of points of the third preset semantic category includes:

[0087] Cluster fitting is performed on the points of the third preset semantic category in the source point cloud and the target point cloud respectively to obtain the second point sets constituting the plane in the source point cloud and the target point cloud respectively, and the points in the second point sets whose flatness is less than the preset flatness threshold are deleted.

[0088] Specifically, points of the third preset semantic category in the source point cloud and the target point cloud may be clustered respectively to obtain at least one planar object instance in the source point cloud and at least one planar object instance in the target point cloud.

[0089] Among them, points belonging to the third preset semantic category in the source point cloud can be screened out, and for the screened points, Euclidean distance clustering is performed on points with the same semantic label, and each type of point obtained by clustering is regarded as a surface object instance.

[0090] Similarly, points belonging to the third preset semantic category in the target point cloud can also be screened out, and for the screened points, Euclidean distance clustering is performed on points with the same semantic label, and each type of point obtained by clustering is regarded as a surface object instance.

[0091] For example, if both the source point cloud and the target point cloud include multiple points with the semantic label "signboard", the surface object instance of the signboard in the source point cloud and the surface object instance of the signboard in the target point cloud can be obtained by clustering.

[0092] Then, for each surface object instance, iterative plane fitting can be performed on the points included in the surface object instance until the number of points included in the surface object instance that are not fitted into a plane is less than a second threshold, and the points included in each fitted plane are respectively used as a second point set.

[0093] In the embodiment of the present disclosure, the RANSAC algorithm can be used for iterative plane fitting. For example, if both the source point cloud and the target point cloud include multiple signboards that are close to each other and approximately on the same plane, the points included in these signboards may be clustered into a surface object instance. Through iterative plane fitting, the plane corresponding to each signboard can be separated from a surface object instance.

[0094] In this way, through the above-mentioned iterative plane fitting process, even if multiple surface objects are mistakenly clustered into one surface object instance, the iterative plane fitting method can be used to separate the plane corresponding to each surface object, which can avoid fitting multiple surface objects with close distances into one plane, thereby accurately deleting points that do not contribute to the geometric constraints, and avoiding the mistaken deletion of points that contribute to the geometric constraints, thereby improving the accuracy of subsequent point cloud registration.

[0095] The points corresponding to the fitted plane must all meet the condition that the distance from each point to the fitted plane is less than a distance threshold. For example, the distance threshold can be 0.1 meter. If a plane is fitted from 500 points, then the distance from each of these 500 points to the plane is less than 0.1 meter. At the same time, the normal vector of the plane can be used as the normal vector for each point in the plane.

[0096] Based on the points corresponding to the above fitted plane, the standard deviation σ of the points included in the plane can be calculated according to the distance from the point to the plane. sign , and use exp{-C sign σ sign}Calculate the flatness of the plane. Where, C sign are the preset parameters.

[0097] In the embodiment of the present disclosure, after the flatness of the plane is calculated using the above formula, the flatness of the plane is used as the flatness of the point corresponding to the plane, and then the flatness of each point can be used as the feature quality score, and the points with a feature quality score less than the preset flatness threshold (s) are deleted. plane ) point.

[0098] Since all points corresponding to a plane have the same flatness, if the flatness of the plane is less than the preset flatness threshold (splane ), all points included in the plane will be deleted.

[0099] For example, the preset flatness threshold (s plane ) is 1. If the flatness of a plane is 0.5, all the points corresponding to the plane can be deleted.

[0100] The preprocessing of points of the fourth preset semantic category includes:

[0101] For each point of the fourth preset semantic category in the source point cloud and the target point cloud, the flatness of each point is calculated based on the covariance matrix of the points in the preset neighborhood range of the point, and points with flatness less than the preset flatness threshold are deleted.

[0102] In the embodiment of the present disclosure, each point of the fourth preset semantic category in the source point cloud and the target point cloud can be traversed, and the normal vector of each point can be approximately calculated using the distribution of points within a preset neighborhood range of each point.

[0103] Specifically, for each point, first search for points within the preset neighborhood of the point, then calculate the covariance matrix of the searched points, perform eigenvalue decomposition based on the covariance matrix, use the eigenvector corresponding to the minimum eigenvalue λ1 obtained by eigenvalue decomposition as the normal vector of the point, and use the flatness (λ2-λ1) / λ3 as the feature quality score of the point, and delete the points whose feature quality score is less than the preset flatness threshold (s plane ) point.

[0104] By preprocessing the points of the third preset semantic category and the fourth preset semantic category, it is possible to avoid the points that do not actually belong to the plane and the points that do not contribute much to the geometric constraints from affecting the point cloud registration results, thereby improving the accuracy of point cloud registration.

[0105] In some embodiments of the present disclosure, in the above S103, based on the position of each point in the source point cloud in the target point cloud coordinate system, searching for the nearest neighbor point with the same semantic category as the point in the target point cloud, and treating the point and the nearest neighbor point corresponding to the point as a related point pair, the related point pairs can also be processed to filter out erroneously related related point pairs. The method specifically includes:

[0106] For each associated point pair, if the associated point pair belongs to the semantic category corresponding to the rod-shaped object, and the distance between the two points included in the associated point pair in the target point cloud coordinate system is greater than the preset distance threshold, or the main direction angle between the two points included in the associated point pair is greater than the first angle threshold, then the associated point pair is deleted.

[0107] For each associated point pair, if the associated point pair belongs to the semantic category corresponding to the planar object, and the distance between the two points included in the associated point pair in the target point cloud coordinate system is greater than the preset distance threshold, or the angle between the normal vectors of the two points included in the associated point pair is greater than the second angle threshold, then the associated point pair is deleted.

[0108] For example, if a pair of associated points includes point A in the source point cloud and point B in the target point cloud, and both point A and point B have the semantic label "telephone pole," if the distance between point A and point B after conversion to the target point cloud's coordinate system is greater than a preset distance threshold, indicating that the two points are far apart in the same coordinate system, it can be determined that point A and point B are not associated. Alternatively, if the main direction angle between point A and point B is greater than a first angle threshold, indicating that the main directions of the lines on which point A and point B lie are different, it can be determined that point A and point B are not associated.

[0109] For another example, if a pair of associated points includes point C in the source point cloud and point D in the target point cloud, and the semantic labels of both points C and D are "signs," if the distance between point C and point D after conversion to the target point cloud's coordinate system is greater than a preset distance threshold, indicating that the two points are far apart in the same coordinate system, it can be determined that points C and D are not associated. Alternatively, if the angle between the normal vectors of points C and D is greater than a second angle threshold, indicating that the normal vectors of the planes in which points C and D lie are different, it can be determined that points C and D are not associated.

[0110] By deleting associated point pairs using the above method, incorrectly associated point pairs can be filtered out, making subsequent point cloud registration results more accurate.

[0111] On the basis of the above embodiment, the final relative pose between the source point cloud and the target point cloud is calculated based on the associated point pairs and geometric information in the source point cloud and the target point cloud, which can be specifically implemented as follows:

[0112] Calculate the relative pose based on the associated point pairs in the source and target point clouds, and determine whether the calculated relative pose has converged. If so, use the calculated relative pose as the final relative pose between the source and target point clouds. If not, use the calculated relative pose as the current relative pose, and return to the steps of transforming the points in the source point cloud to the target point cloud coordinate system based on the current relative pose until the final relative pose is obtained.

[0113] Using this method, the relative pose of the source point cloud and the target point cloud can be calculated using the associated point pairs. If the relative pose does not converge, the associated point pairs can be reconstructed based on the relative pose calculated this time. Through iteration, the accurate relative pose between the source point cloud and the target point cloud can be gradually calculated, which can improve the accuracy of the calculated final relative pose.

[0114] Among them, the method for calculating the relative pose is:

[0115] It is determined whether the geometric constraints represented by the associated point pairs whose semantic categories belong to rod-shaped objects and the associated point pairs whose semantic categories belong to planar objects are sufficient.

[0116] If so, the relative pose is calculated based on the associated point pairs whose semantic category belongs to the rod-shaped object and the associated point pairs whose semantic category belongs to the planar object.

[0117] If not, the relative pose is calculated based on the associated point pairs whose semantic categories belong to rod-shaped objects, the associated point pairs whose semantic categories belong to planar objects, and the associated point pairs whose semantic categories belong to ground markers.

[0118] For points belonging to rod-shaped objects, points whose main direction is parallel to the x-axis can provide translation constraints in the y-axis and z-axis directions, points whose main direction is parallel to the y-axis can provide translation constraints in the x-axis and z-axis directions, and points whose main direction is parallel to the z-axis can provide translation constraints in the x-axis and y-axis directions.

[0119] Based on the above principle, for a pair of connected points belonging to a rod, |o·X| can be used as the constraint contribution of each point to the y-axis and z-axis, |o·Y| as the constraint contribution of each point to the x-axis and z-axis, and |o·Z| as the constraint contribution of each point to the x-axis and y-axis. Here, o represents the main direction of the point. X, Y, and Z are the unit vectors in the x-axis, y-axis, and z-axis directions, respectively.

[0120] For the points in the associated point pairs belonging to the rod, the |o·Y| and |o·Z| of all points are summed to obtain the constraint contribution of the associated point pairs belonging to the rod in the x-axis direction; the |o·X| and |o·Z| of all points are summed to obtain the constraint contribution of the associated point pairs belonging to the rod in the y-axis direction; the |o·X| and |o·Y| of all points are summed to obtain the constraint contribution of the associated point pairs belonging to the rod in the z-axis direction.

[0121] Then, the constraint contribution in the x-axis direction, the constraint contribution in the y-axis direction, and the constraint contribution in the z-axis direction are added together to calculate the total geometric constraint contribution value of the associated point pair belonging to the rod-shaped object.

[0122] For points belonging to planar objects, points whose normal vectors are parallel to the x-axis can provide translation constraints in the x-axis direction, points whose normal vectors are parallel to the y-axis can provide translation constraints in the y-axis direction, and points whose normal vectors are parallel to the z-axis can provide translation constraints in the z-axis direction.

[0123] Based on the above principle, for the associated point pairs belonging to the planar object, |n·X| can be used as the translation constraint of each point pair in the x-axis direction, |n·Y| can be used as the translation constraint of each point in the y-axis direction, and |n·Z| can be used as the translation constraint of each point in the y-axis direction, where n is the normal vector of the point.

[0124] For the points in the associated point pairs belonging to the surface object, the |n·X| of all points is summed to obtain the constraint contribution of the associated point pairs belonging to the surface object in the x-axis direction; the |n·Y| of all points is summed to obtain the constraint contribution of the associated point pairs belonging to the surface object in the y-axis direction; the |n·Z| of all points is summed to obtain the constraint contribution of the associated point pairs belonging to the surface object in the z-axis direction.

[0125] Then, the constraint contribution in the x-axis direction, the constraint contribution in the y-axis direction, and the constraint contribution in the z-axis direction are added together to calculate the total geometric constraint contribution value of the associated point pair belonging to the planar object.

[0126] The total geometric constraint contribution values ​​of the rod-shaped point pairs and the surface-shaped point pairs are summed to obtain the total geometric constraint contribution value of all the point pairs. If this total geometric constraint contribution value is greater than or equal to the constraint threshold, the geometric constraints represented by the rod-shaped point pairs and the surface-shaped point pairs are considered sufficient. Otherwise, the geometric constraints are considered insufficient.

[0127] Through the above method, it is possible to accurately calculate whether the geometric constraints of the associated point pairs belonging to rod-shaped objects and the associated point pairs belonging to surface objects are sufficient. When the geometric constraints are insufficient, the associated point pairs of ground markers commonly present in urban scenes (such as crosswalks, turn arrows, etc.) can be further used to calculate the relative pose. In this way, the associated point pairs of ground markers are used to provide additional geometric constraints, thereby improving the accuracy of the calculated relative pose and thus improving the robustness of point cloud registration.

[0128] When it is determined that the geometric constraints are sufficient, the relative pose can be calculated based on the objective function corresponding to the rod-shaped object and the objective function corresponding to the planar object.

[0129] Among them, the objective function of the rod is the point-to-line objective function, and the specific formula is:

[0130] d pole (i)=‖o i ×(q i -Rp i -t)‖

[0131] Among them, i is the main direction of the point cloud, p i and q i is a pair of associated points, pi is a point in the source point cloud, q i is a point in the target point cloud, R∈SO(3), R is the rotation matrix, SO(3) is the three-dimensional rotation group, t is the translation vector, It is a three-dimensional Euclidean space.

[0132] The objective function of a surface object is a point-to-surface objective function, and the specific formula is:

[0133] d plane (i)=|n i ·(q i -Rp i -t)|

[0134] Among them, n i is the normal vector of the point.

[0135] Correspondingly, the relative pose is:

[0136]

[0137] Among them, R * is the calculated rotation matrix, t * is the calculated translation matrix, argmin refers to the set of all independent variables that make the function obtain the minimum value, N pole is the number of associated point pairs of the rod, N plane is the number of associated point pairs of the surface object.

[0138] When it is determined that the geometric constraints are insufficient, the relative pose can be calculated based on the objective function corresponding to the rod-shaped object, the objective function corresponding to the planar object, and the objective function corresponding to the ground marker.

[0139] Among them, the objective function corresponding to the ground marker is the objective function of the distance between the marker points, and the specific formula is:

[0140] d point (i)=‖q i -Rp i -t‖

[0141] Correspondingly, the relative pose is:

[0142]

[0143] N point is the number of associated point pairs of ground markers.

[0144] By adopting this method, when the geometric constraints are insufficient, ground markers can be used to assist in point cloud registration, which can avoid the problem of solution degradation caused by insufficient geometric constraints and further improve the accuracy of point cloud registration.

[0145] In the above embodiment, the method for determining whether the relative posture has converged is:

[0146] If the difference between the rotation angle of the calculated relative pose and the rotation matrix in the current relative pose is less than the third angle threshold, and the difference in the translation distance of the translation matrix is ​​less than the translation distance threshold, then the calculated relative pose is determined to converge; otherwise, it is determined that the calculated relative pose does not converge.

[0147] When the relative poses obtained from the two calculations are relatively close, it can be determined that the calculated relative poses converge, which can improve the calculation efficiency while ensuring accuracy, and can obtain a more accurate relative pose, thereby improving the accuracy of point cloud registration.

[0148] Based on the above embodiments, the present disclosure provides an exemplary flow chart of a point cloud registration method, as shown in FIG. Figure 3 As shown, the method includes:

[0149] Perform semantic segmentation on the source point cloud and the target point cloud to obtain the semantic category of each point in the source point cloud and the target point cloud.

[0150] The source point cloud and target point cloud are preprocessed according to the semantic category of each point.

[0151] The pre-processed source point cloud and target point cloud are associated to determine the associated point pairs of the source point cloud and the target point cloud.

[0152] like Figure 3 As shown, as an example, an associated point pair can be established between the points with the semantic category of telephone poles in the source point cloud and the points with the semantic category of telephone poles in the target point cloud. Similarly, associated point pairs can be established for curbs, lane lines, and other poles respectively.

[0153] Similarly, for surface objects, associated point pairs can be established for walls, floors, signboards, fences, and other surfaces.

[0154] Then, the relative pose of the source point cloud and the target point cloud is solved based on the associated point pairs.

[0155] If the relative pose obtained by the solution converges, the relative pose is output as the final relative pose. If the relative pose obtained by the solution does not converge, the relative pose is used as the current relative pose and the process of establishing associated point pairs is returned until the calculated relative pose converges.

[0156] By using the above method, after pre-processing the source point cloud and the target point cloud, the point clouds in the source point cloud and the target point cloud that cannot provide geometric constraints are deleted, which can improve the accuracy of point cloud registration and the efficiency of subsequent point cloud association. When performing point cloud association, only determining the associated point cloud in the target point cloud of the same semantic category as the source point cloud can improve the association efficiency. Compared with directly associating the source point cloud and the target point cloud after transforming the points in the source point cloud to the target point cloud coordinate system, it can also reduce misassociation. When solving relative pose, if the current geometric constraints of the rod-shaped object and the surface object are insufficient, the relative pose can be solved by using the associated point pairs of the ground markers based on the associated point pairs of the rod-shaped object and the surface object. In this way, the problem of solution degradation in the scene of insufficient geometric constraints can be solved, and the point cloud registration error is reduced.

[0157] Based on the same inventive concept, the embodiment of the present disclosure provides a device for high-precision map point cloud registration, such as Figure 4 As shown, the device includes:

[0158] The semantic segmentation module 401 is used to perform semantic segmentation on the source point cloud and the target point cloud respectively to obtain the semantic category of each point in the source point cloud and the target point cloud.

[0159] The coordinate transformation module 402 is used to use the initial relative pose as the current relative pose and transform the points in the source point cloud to the target point cloud coordinate system based on the current relative pose; the initial relative pose is the pre-measured relative pose between the source point cloud and the target point cloud.

[0160] The point cloud association module 403 searches for the nearest neighbor point with the same semantic category as that of each point in the source point cloud based on the position of the point in the target point cloud coordinate system, and regards the point and its corresponding nearest neighbor point as an association point pair.

[0161] The calculation module 404 is configured to calculate the final relative pose between the source point cloud and the target point cloud based on the associated point pairs and set information in the source point cloud and the target point cloud.

[0162] Optionally, the calculation module 404 is specifically configured to:

[0163] The relative pose is calculated based on the associated point pairs in the source point cloud and the target point cloud, and whether the calculated relative pose converges is determined.

[0164] If converged, the calculated relative pose is used as the final relative pose between the source point cloud and the target point cloud.

[0165] If convergence is not achieved, the calculated relative pose is used as the current relative pose, triggering the coordinate transformation module to transform the points in the source point cloud to the target point cloud coordinate system based on the current relative pose until the final relative pose is obtained.

[0166] Optionally, the device further comprises:

[0167] The rod-shaped object preprocessing module is used to delete points whose straightness is less than a preset straightness threshold based on the straightness of each point in the source point cloud and the target point cloud whose semantic category belongs to the rod-shaped object.

[0168] The surface object preprocessing module is used to delete points whose flatness is less than a preset flatness threshold based on the flatness of each point whose semantic category belongs to the surface object in the source point cloud and the target point cloud.

[0169] The ground marker preprocessing module is used to perform density filtering on points in the source point cloud and the target point cloud whose semantic categories belong to ground markers.

[0170] The dynamic point cloud preprocessing module performs ray tracing on each point in the source point cloud and the target point cloud whose semantic category belongs to the dynamic category, determines the dynamic and static state of each point, and deletes the points in motion.

[0171] Optionally, the rod-shaped object pre-processing module is specifically used to:

[0172] Cluster fitting is performed on the points of the first preset semantic category in the source point cloud and the target point cloud respectively, to obtain the first point set constituting the straight line of the rod-shaped object in the source point cloud and the target point cloud respectively; the first preset semantic category includes the semantic category corresponding to the rod-shaped object, which can be fitted into a straight line.

[0173] Delete points in the first point set whose straightness is less than a preset straightness threshold.

[0174] For each point of the second preset semantic category in the source point cloud and the target point cloud, the straightness of each point is calculated based on the covariance matrix of the points in the preset neighborhood range of the point, and the points with straightness less than the preset straightness threshold are deleted; the second preset semantic category includes the semantic categories corresponding to rod-shaped objects and the semantic categories that cannot be fitted as straight lines.

[0175] The surface preprocessing module is specifically used for:

[0176] Cluster fitting is performed on the points of the third preset semantic category in the source point cloud and the target point cloud respectively to obtain a second point set constituting a plane in the source point cloud and the target point cloud respectively; the third preset semantic category includes the semantic category corresponding to the planar object, which can be fitted into a semantic category of a plane.

[0177] Delete points in the second point set whose flatness is less than a preset flatness threshold.

[0178] For each point of the fourth preset semantic category in the source point cloud and the target point cloud, the flatness of each point is calculated based on the covariance matrix of the points within the preset neighborhood of the point, and the points with flatness less than the preset flatness threshold are deleted; the fourth preset semantic category includes the semantic categories corresponding to planar objects, which cannot be fitted into a plane.

[0179] Optionally, the rod-shaped object pre-processing module is specifically used to:

[0180] Points of a first preset semantic category in the source point cloud and the target point cloud are clustered respectively to obtain at least one rod instance in the source point cloud and at least one rod instance in the target point cloud.

[0181] For each rod instance, iterative straight line fitting is performed on the points included in the rod instance until the number of points in the rod instance that are not fitted into straight lines is less than a first threshold, and the points included in each straight line obtained by fitting are respectively regarded as a first point set.

[0182] The surface preprocessing module is specifically used for:

[0183] Points of a third preset semantic category in the source point cloud and the target point cloud are clustered respectively to obtain at least one planar object instance in the source point cloud and at least one planar object instance in the target point cloud.

[0184] For each planar object instance, iterative plane fitting is performed on the points included in the planar object instance until the number of points included in the planar object instance that are not fitted into a plane is less than a second threshold, and the points included in each fitted plane are respectively used as a second point set.

[0185] Optionally, the device further includes a deletion module, which is configured to:

[0186] For each associated point pair, if the associated point pair belongs to the semantic category corresponding to the rod-shaped object, and the distance between the two points included in the associated point pair in the target point cloud coordinate system is greater than the preset distance threshold, or the main direction angle between the two points included in the associated point pair is greater than the first angle threshold, then the associated point pair is deleted.

[0187] For each associated point pair, if the associated point pair belongs to the semantic category corresponding to the planar object, and the distance between the two points included in the associated point pair in the target point cloud coordinate system is greater than the preset distance threshold, or the angle between the normal vectors of the two points included in the associated point pair is greater than the second angle threshold, then the associated point pair is deleted.

[0188] Optionally, the calculation module 404 is specifically configured to:

[0189] It is determined whether the geometric constraints represented by the associated point pairs whose semantic categories belong to rod-shaped objects and the associated point pairs whose semantic categories belong to planar objects are sufficient.

[0190] If so, the relative pose is calculated based on the associated point pairs whose semantic category belongs to the rod-shaped object and the associated point pairs whose semantic category belongs to the planar object.

[0191] If not, the relative pose is calculated based on the associated point pairs whose semantic categories belong to rod-shaped objects, the associated point pairs whose semantic categories belong to planar objects, and the associated point pairs whose semantic categories belong to ground markers.

[0192] Optionally, the calculation module 404 is specifically configured to:

[0193] If the difference between the rotation angle of the calculated relative pose and the rotation matrix in the current relative pose is less than the third angle threshold, and the difference in the translation distance of the translation matrix is ​​less than the translation distance threshold, then the calculated relative pose is determined to converge; otherwise, it is determined that the calculated relative pose does not converge.

[0194] In the technical solutions disclosed herein, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0195] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0196] Figure 5 A schematic block diagram of an example electronic device 500 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are provided as examples only and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0197] like Figure 5 As shown, the device 500 includes a computing unit 501, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 502 or a computer program loaded from a storage unit 508 into a random access memory (RAM) 503. Various programs and data required for the operation of the device 500 can also be stored in the RAM 503. The computing unit 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0198] Various components in device 500 are connected to I / O interface 505, including: an input unit 506, such as a keyboard, mouse, etc.; an output unit 507, such as various types of displays, speakers, etc.; a storage unit 508, such as a magnetic disk, optical disk, etc.; and a communication unit 509, such as a network card, modem, wireless communication transceiver, etc. The communication unit 509 allows device 500 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0199] The computing unit 501 can be a variety of general-purpose and / or specialized processing components with processing and computing capabilities. Some examples of the computing unit 501 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 501 performs the various methods and processes described above, such as the method for HD map point cloud registration. For example, in some embodiments, the method for HD map point cloud registration can be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 508. In some embodiments, part or all of the computer program can be loaded and / or installed onto the device 500 via the ROM 502 and / or the communication unit 509. When the computer program is loaded into the RAM 503 and executed by the computing unit 501, one or more steps of the method for HD map point cloud registration described above can be performed. Alternatively, in other embodiments, the computing unit 501 may be configured to perform the method for high-precision map point cloud registration in any other appropriate manner (for example, by means of firmware).

[0200] Various embodiments of the systems and techniques described above can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0201] The program code for implementing the method of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device so that when the program code is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0202] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0203] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0204] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0205] A computer system may include a client and a server. The client and server are generally remote from each other and typically interact through a communication network. The client-server relationship arises through computer programs running on the respective computers and having a client-server relationship with each other. The server may be a cloud server, a server in a distributed system, or a server integrated with a blockchain.

[0206] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved. This is not limited herein.

[0207] The above specific embodiments do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure shall be included within the scope of protection of this disclosure.

Claims

1. A method for high-precision map point cloud registration, comprising: Perform semantic segmentation on the source point cloud and the target point cloud respectively to obtain the semantic category of each point in the source point cloud and the target point cloud; Taking the initial relative pose as the current relative pose, and transforming the points in the source point cloud to the target point cloud coordinate system based on the current relative pose; The initial relative pose is a pre-measured relative pose between the source point cloud and the target point cloud; Based on the position of each point in the source point cloud in the target point cloud coordinate system, searching for the nearest neighbor point in the target point cloud with the same semantic category as the point, and treating the point and its corresponding nearest neighbor point as an associated point pair; Calculating a final relative pose between the source point cloud and the target point cloud based on associated point pairs and geometric information in the source point cloud and the target point cloud; The step of calculating the final relative pose between the source point cloud and the target point cloud based on the associated point pairs and geometric information in the source point cloud and the target point cloud comprises: Calculating a relative pose based on associated point pairs in the source point cloud and the target point cloud, and determining whether the calculated relative pose converges; If converged, the calculated relative pose is used as the final relative pose between the source point cloud and the target point cloud; If convergence has not occurred, the calculated relative pose is used as the current relative pose, and the process of transforming the points in the source point cloud into the target point cloud coordinate system based on the current relative pose is returned to, until the final relative pose is obtained. The calculating the relative pose based on the associated point pairs in the source point cloud and the target point cloud includes: Determine whether the geometric constraints represented by the associated point pairs belonging to the semantic category of rod-shaped objects and the associated point pairs belonging to the semantic category of surface-shaped objects are sufficient; If so, the relative pose is calculated based on the associated point pairs whose semantic categories belong to rod-shaped objects and the associated point pairs whose semantic categories belong to surface-shaped objects; If not, the relative pose is calculated based on the associated point pairs whose semantic categories belong to rod-shaped objects, the associated point pairs whose semantic categories belong to planar objects, and the associated point pairs whose semantic categories belong to ground markers.

2. The method according to claim 1, after performing semantic segmentation on the source point cloud and the target point cloud to obtain the semantic category of each point in the source point cloud and the target point cloud, the method further comprises: Based on the straightness of each point in the source point cloud and the target point cloud whose semantic category belongs to the rod-shaped object, deleting points whose straightness is less than a preset straightness threshold; Based on the flatness of each point in the source point cloud and the target point cloud, whose semantic category belongs to planar objects, deleting points whose flatness is less than a preset flatness threshold; Performing density filtering on points in the source point cloud and the target point cloud whose semantic categories belong to ground markers; Ray tracing is performed on each point in the source point cloud and the target point cloud whose semantic category belongs to the dynamic category, the dynamic and static states of each point are determined, and points in a moving state are deleted.

3. The method according to claim 2, wherein: The step of deleting points having a straightness less than a preset straightness threshold based on the straightness of each point in the source point cloud and the target point cloud whose semantic category belongs to the rod-shaped object comprises: performing cluster fitting on points of a first preset semantic category in the source point cloud and the target point cloud, respectively, to obtain first point sets constituting a straight line of the rod-shaped object in the source point cloud and the target point cloud, respectively; the first preset semantic category includes semantic categories corresponding to the rod-shaped object that can be fitted into a straight line; Deleting points in the first point set whose straightness is less than a preset straightness threshold; For each point of a second preset semantic category in the source point cloud and the target point cloud, the straightness of each point is calculated based on the covariance matrix of points within a preset neighborhood of the point, and points with a straightness less than a preset straightness threshold are deleted; the second preset semantic category includes semantic categories corresponding to the rod-shaped object that cannot be fitted as a straight line; The step of deleting points having a flatness less than a preset flatness threshold based on the flatness of each point in the source point cloud and the target point cloud whose semantic category belongs to a planar object comprises: performing cluster fitting on points of a third preset semantic category in the source point cloud and the target point cloud, respectively, to obtain a second set of points constituting a plane in the source point cloud and the target point cloud, respectively; the third preset semantic category includes a semantic category that can be fitted into a plane among the semantic categories corresponding to the planar object; Deleting points in the second point set whose flatness is less than a preset flatness threshold; For each point of a fourth preset semantic category in the source point cloud and the target point cloud, the flatness of each point is calculated based on the covariance matrix of the points within a preset neighborhood of the point, and points whose flatness is less than the preset flatness threshold are deleted; the fourth preset semantic category includes the semantic categories corresponding to the planar object that cannot be fitted into a plane.

4. The method according to claim 3, wherein: The clustering and fitting of the points of the first preset semantic category in the source point cloud and the target point cloud is performed to obtain a first set of points constituting a straight line of the rod in the source point cloud and the target point cloud, respectively, including: Clustering points of a first preset semantic category in the source point cloud and the target point cloud respectively to obtain at least one rod instance in the source point cloud and at least one rod instance in the target point cloud; For each rod instance, iterative straight line fitting is performed on the points included in the rod instance until the number of points included in the rod instance that are not fitted into straight lines is less than a first threshold, and the points included in each straight line obtained by fitting are respectively regarded as a first point set; The performing cluster fitting on the points of the third preset semantic category in the source point cloud and the target point cloud respectively to obtain a second set of points constituting a plane in the source point cloud and the target point cloud respectively includes: Clustering points of a third preset semantic category in the source point cloud and the target point cloud respectively to obtain at least one planar object instance in the source point cloud and at least one planar object instance in the target point cloud; For each planar object instance, iterative plane fitting is performed on the points included in the planar object instance until the number of points included in the planar object instance that are not fitted into a plane is less than a second threshold, and the points included in each fitted plane are respectively used as a second point set.

5. The method according to claim 1, further comprising: searching for a nearest neighbor point of the same semantic category as each point in the source point cloud based on the position of the point in the target point cloud coordinate system, and treating the point and its corresponding nearest neighbor point as a connected point pair; For each associated point pair, if the associated point pair belongs to the semantic category corresponding to the rod-shaped object, and the distance between the two points included in the associated point pair in the target point cloud coordinate system is greater than the preset distance threshold, or the main direction angle between the two points included in the associated point pair is greater than the first angle threshold, then the associated point pair is deleted; For each associated point pair, if the associated point pair belongs to the semantic category corresponding to the planar object, and the distance between the two points included in the associated point pair in the target point cloud coordinate system is greater than the preset distance threshold, or the angle between the normal vectors of the two points included in the associated point pair is greater than the second angle threshold, then the associated point pair is deleted.

6. The method according to claim 1, wherein The determining whether the calculated relative posture converges includes: If the difference between the calculated relative posture and the rotation angle of the rotation matrix in the current relative posture is less than the third angle threshold, and the difference in the translation distance of the translation matrix is ​​less than the translation distance threshold, then it is determined that the calculated relative posture converges; otherwise, it is determined that the calculated relative posture does not converge.

7. A device for high-precision map point cloud registration, comprising: The semantic segmentation module is used to perform semantic segmentation on the source point cloud and the target point cloud respectively to obtain the semantic category of each point in the source point cloud and the target point cloud; a coordinate transformation module, configured to transform the points in the source point cloud into a target point cloud coordinate system based on an initial relative pose as a current relative pose; the initial relative pose being a pre-measured relative pose between the source point cloud and the target point cloud; A point cloud association module searches for a nearest neighbor point in the target point cloud with the same semantic category as each point in the source point cloud based on the position of the point in the target point cloud coordinate system, and treats the point and its corresponding nearest neighbor point as an association point pair; A calculation module, configured to calculate a final relative pose between the source point cloud and the target point cloud based on associated point pairs and set information in the source point cloud and the target point cloud; The computing module is specifically configured to: Calculating a relative pose based on associated point pairs in the source point cloud and the target point cloud, and determining whether the calculated relative pose converges; If converged, the calculated relative pose is used as the final relative pose between the source point cloud and the target point cloud; If convergence has not occurred, the calculated relative pose is used as the current relative pose, and the coordinate transformation module is triggered to transform the points in the source point cloud into the target point cloud coordinate system based on the current relative pose until the final relative pose is obtained; The computing module is specifically configured to: Determine whether the geometric constraints represented by the associated point pairs belonging to the semantic category of rod-shaped objects and the associated point pairs belonging to the semantic category of surface-shaped objects are sufficient; If so, the relative pose is calculated based on the associated point pairs whose semantic categories belong to rod-shaped objects and the associated point pairs whose semantic categories belong to surface-shaped objects; If not, the relative pose is calculated based on the associated point pairs whose semantic categories belong to rod-shaped objects, the associated point pairs whose semantic categories belong to planar objects, and the associated point pairs whose semantic categories belong to ground markers.

8. The apparatus according to claim 7, further comprising: a rod-shaped object preprocessing module, configured to delete points whose straightness is less than a preset straightness threshold based on the straightness of each point in the source point cloud and the target point cloud whose semantic category belongs to the rod-shaped object; A planar object preprocessing module is configured to delete points whose planarity is less than a preset planarity threshold based on the planarity of each point in the source point cloud and the target point cloud whose semantic category belongs to the planar object; A ground marker preprocessing module, configured to perform density filtering on points in the source point cloud and the target point cloud whose semantic categories belong to ground markers; The dynamic point cloud preprocessing module performs ray tracing on each point in the source point cloud and the target point cloud whose semantic category belongs to the dynamic category, determines the dynamic and static state of each point, and deletes points in a moving state.

9. The device according to claim 8, wherein The rod-shaped object preprocessing module is specifically used for: Performing cluster fitting on points of a first preset semantic category in the source point cloud and the target point cloud, respectively, to obtain first point sets constituting a straight line of a rod-shaped object in the source point cloud and the target point cloud, respectively; The first preset semantic category includes a semantic category that can be fitted into a straight line among the semantic categories corresponding to the rod-shaped object; Deleting points in the first point set whose straightness is less than a preset straightness threshold; For each point of a second preset semantic category in the source point cloud and the target point cloud, the straightness of each point is calculated based on the covariance matrix of points within a preset neighborhood of the point, and points with a straightness less than a preset straightness threshold are deleted; the second preset semantic category includes semantic categories corresponding to the rod-shaped object that cannot be fitted as a straight line; The surface object pre-processing module is specifically used for: performing cluster fitting on points of a third preset semantic category in the source point cloud and the target point cloud, respectively, to obtain a second set of points constituting a plane in the source point cloud and the target point cloud, respectively; The third preset semantic category includes a semantic category that can be fitted into a plane among the semantic categories corresponding to the planar object; Deleting points in the second point set whose flatness is less than a preset flatness threshold; For each point of a fourth preset semantic category in the source point cloud and the target point cloud, the flatness of each point is calculated based on the covariance matrix of the points within a preset neighborhood of the point, and points whose flatness is less than the preset flatness threshold are deleted; the fourth preset semantic category includes the semantic categories corresponding to the planar object that cannot be fitted into a plane.

10. The device according to claim 9, wherein The rod-shaped object preprocessing module is specifically used for: Clustering points of a first preset semantic category in the source point cloud and the target point cloud respectively to obtain at least one rod instance in the source point cloud and at least one rod instance in the target point cloud; For each rod instance, iterative straight line fitting is performed on the points included in the rod instance until the number of points included in the rod instance that are not fitted into straight lines is less than a first threshold, and the points included in each straight line obtained by fitting are respectively regarded as a first point set; The surface object pre-processing module is specifically used for: Clustering points of a third preset semantic category in the source point cloud and the target point cloud respectively to obtain at least one planar object instance in the source point cloud and at least one planar object instance in the target point cloud; For each planar object instance, iterative plane fitting is performed on the points included in the planar object instance until the number of points included in the planar object instance that are not fitted into a plane is less than a second threshold, and the points included in each fitted plane are respectively used as a second point set.

11. The apparatus according to claim 7, further comprising a deletion module; The deletion module is used to: For each associated point pair, if the associated point pair belongs to the semantic category corresponding to the rod-shaped object, and the distance between the two points included in the associated point pair in the target point cloud coordinate system is greater than the preset distance threshold, or the main direction angle between the two points included in the associated point pair is greater than the first angle threshold, then the associated point pair is deleted; For each associated point pair, if the associated point pair belongs to the semantic category corresponding to the planar object, and the distance between the two points included in the associated point pair in the target point cloud coordinate system is greater than the preset distance threshold, or the angle between the normal vectors of the two points included in the associated point pair is greater than the second angle threshold, then the associated point pair is deleted.

12. The device according to claim 7, wherein The computing module is specifically configured to: If the difference between the calculated relative posture and the rotation angle of the rotation matrix in the current relative posture is less than the third angle threshold, and the difference in the translation distance of the translation matrix is ​​less than the translation distance threshold, then it is determined that the calculated relative posture converges; otherwise, it is determined that the calculated relative posture does not converge.

13. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 6.

14. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 6.

15. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 6.

Images