Point cloud registration method and edge server

Abstract

This invention provides a point cloud registration method and an edge server. The point cloud registration method includes: performing coarse registration on a source point cloud and a target point cloud to obtain a coarsely registered point cloud of the target point cloud, wherein the source point cloud is a point cloud obtained by measurement using a first measuring device, and the target point cloud is a point cloud obtained by measurement using a second measuring device, the first measuring device and the second measuring device having a common measurement area; and performing precise registration on the source point cloud and the coarsely registered point cloud. Specifically, when performing coarse registration on the source point cloud and the target point cloud, a method combining depth map features and feature point distribution features is used. Since converting the point cloud into a depth map maps the point cloud data to a planar coordinate system, the density of denser point clouds becomes sparser, which can provide better registration results even when the density difference between the two point clouds to be registered is large.

Description

Technical Field

[0001] This invention relates to the field of point cloud stitching technology, and more particularly to a point cloud registration method and an edge server. Background Technology

[0002] In roadside sensing scenarios, LiDAR (Light Detection and Ranging) is typically used to acquire point cloud data of the road surface. LiDARs are arranged at fixed intervals. By registering the shared point cloud information of different LiDARs in a common measurement area, the coordinate system transformation relationship between the different LiDARs can be obtained, enabling cross-device sensing and tracking. The registration effect between point clouds directly affects the cross-device sensing and tracking of targets. However, due to the varying distances of the radars, radars closer to the common measurement area have higher point cloud density, while radars further away have lower point cloud density in the common measurement area. This can easily lead to poor registration results during point cloud feature extraction. Summary of the Invention

[0003] This invention provides a point cloud registration method and an edge server to solve the problem of poor registration results caused by large differences in point cloud density obtained from different measurement devices.

[0004] To solve the above-mentioned technical problems, the present invention is implemented as follows:

[0005] In a first aspect, embodiments of the present invention provide a point cloud registration method, including:

[0006] Coarse registration is performed on the source point cloud and the target point cloud to obtain a coarsely registered point cloud of the target point cloud. The source point cloud is a point cloud obtained by measuring with a first measuring device, and the target point cloud is a point cloud obtained by measuring with a second measuring device. The first measuring device and the second measuring device have a common measurement area.

[0007] Perform precise registration between the source point cloud and the coarse registration point cloud;

[0008] The process of coarsely registering the source point cloud and the target point cloud to obtain the coarsely registered point cloud of the target point cloud includes:

[0009] Obtain point cloud depth maps of the source point cloud and the target point cloud;

[0010] Feature points are obtained from the depth maps of the source point cloud and the target point cloud, and the feature points are calculated to obtain a feature point distribution feature description of the source point cloud and the target point cloud.

[0011] Based on the feature points and the feature point distribution feature description, determine the feature point matching relationship between the source point cloud and the target point cloud;

[0012] Based on the feature point matching relationship, the transformation matrix of the target point cloud is obtained;

[0013] Based on the transformation matrix, a coarse registration point cloud is obtained.

[0014] Optionally, before obtaining the point cloud depth map of the target point cloud, the method further includes:

[0015] Based on the position information and angular extrinsic parameter calibration results of the source point cloud and the target point cloud, the target point cloud is initially transformed to obtain the initial transformed point cloud of the target point cloud;

[0016] Point cloud correction is performed on the preliminary transformed point cloud to obtain a coarse transformed point cloud corresponding to the target point cloud, thereby updating the target point cloud.

[0017] Optionally, performing point cloud correction on the preliminary transformed point cloud to obtain the coarse transformed point cloud corresponding to the target point cloud includes:

[0018] Ground point cloud fitting is performed on the source point cloud and the preliminary transformed point cloud respectively to obtain the ground vectors of the source point cloud and the preliminary transformed point cloud;

[0019] The rotation angle of the preliminary transformed point cloud relative to the source point cloud is obtained based on the ground vector;

[0020] The initial converted point cloud is corrected based on the rotation angle to obtain the target point cloud.

[0021] Optionally, ground point cloud fitting can be performed using the following formula:

[0022] Ax + By + Cz + D = 0

[0023] Where A, B, C, and D are constants, and x, y, and z are the location information of the point cloud data.

[0024] Optionally, the rotation angle can be calculated using the following formula:

[0025] θ = arccos(R, V)

[0026] Wherein, θ is the rotation angle, R is the ground vector of the initial converted point cloud, and V is the ground vector of the source point cloud.

[0027] Optionally, the preliminary transformed point cloud can be corrected using the following formula:

[0028] P ti =TP t1

[0029]

[0030] Among them, Pti For the target point cloud, P t1 Let T be the initial transformed point cloud, and T be the transformation matrix.

[0031] Optionally, obtaining the point cloud depth maps of the source point cloud and the target point cloud includes:

[0032] Point cloud clusters are aggregated for the source point cloud and the target point cloud respectively to obtain point cloud clusters of the source point cloud and the target point cloud, and the recognition box of each point cloud cluster is obtained.

[0033] The identification box of the point cloud cluster is voxelized to obtain multiple voxels. The centroid of the midpoint of each voxel is taken as the voxel center point. The N voxel center points closest to the voxel center point form N curvature surfaces. The surface normal vectors of the N curvature surfaces are calculated. The curvature information of the neighborhood surface of the voxel center point is obtained based on the surface normal vectors.

[0034] Based on the curvature information, the similarity between each point cloud cluster in the source point cloud and each point cloud cluster in the target point cloud is calculated. Based on the similarity, the most matching point cloud cluster and the most matching region in the target point cloud and the source point cloud are determined. The point cloud cluster with a larger number of voxels in the most matching region in the source point cloud or the target point cloud is replaced by a point cloud cluster with a smaller number of voxels.

[0035] A point cloud depth map of the source point cloud is generated based on the point cloud clusters of the source point cloud.

[0036] A point cloud depth map of the target point cloud is generated based on the point cloud clusters of the target point cloud.

[0037] Optional, N equals 3.

[0038] Optionally, obtaining the curvature information of the surface surrounding the voxel center point based on the surface normal vector includes:

[0039] The average value of the N surface normal vectors is used as the curvature information of the surface in the neighborhood of the voxel center point.

[0040] Optionally, calculating the similarity between each point cloud cluster in the source point cloud and each point cloud cluster in the target point cloud based on the curvature information includes:

[0041] The voxel space with a smaller number of voxels in the voxel space corresponding to the source point cloud and the target point cloud is selected as the voxel convolution kernel. It is compared with the voxel space with a larger number of voxels to obtain the voxel coefficient of the same corresponding space.

[0042] Based on the curvature information corresponding to the source point cloud and the target point cloud, calculate the curvature similarity coefficient between the source point cloud and the target point cloud;

[0043] The similarity between the point cloud clusters in the source point cloud and the point cloud clusters in the target point cloud is calculated based on the corresponding space occupancy voxel coefficient and the curvature similarity coefficient.

[0044] Optionally, the corresponding space-occupying voxel coefficient can be calculated using the following formula:

[0045]

[0046]

[0047] Where C0 is the voxel coefficient of the corresponding space occupancy, V min V represents the number of voxels in the voxel space of the point cloud with the smaller number of voxels. max n is the number of voxels in the point cloud voxel space with a larger number of voxels, and n is the number of voxels with the same attribute in the voxel convolution kernel and the convolution region. The same attribute refers to the point cloud voxel space that is either point or no point.

[0048] Optionally, the curvature similarity coefficient can be calculated using the following formula:

[0049]

[0050]

[0051] Where c is the curvature similarity coefficient, curvature i For the curvature information corresponding to the source point cloud, curvature j For the curvature information corresponding to the target point cloud, α i α is an attribute of the voxels of the source point cloud. j The attribute of the voxels of the target point cloud is whether the voxel contains points or not.

[0052] Optionally, the similarity between point cloud clusters in the source point cloud and point cloud clusters in the target point cloud can be calculated using the following formula:

[0053] S=c*C0

[0054] Wherein, S is the similarity between the point cloud clusters in the source point cloud and the point cloud clusters in the target point cloud, c is the curvature similarity coefficient, and C0 is the voxel coefficient of the same corresponding space.

[0055] Optionally, generating a point cloud depth map of the source point cloud based on the point cloud clusters of the source point cloud includes:

[0056] Calculate the centroid of each point cloud cluster in the source point cloud, and use the centroid of the point cloud cluster to represent the point cloud cluster;

[0057] The directions of the eigenvectors corresponding to the two larger eigenvalues ​​among the three directions of the centroid are taken as the two axis directions of the coordinate system plane;

[0058] Map the centroid of each point cloud cluster in the source point cloud to a planar grid in the coordinate system plane;

[0059] Calculate the depth information of each grid point, and obtain the point cloud depth map of the source point cloud based on the depth information.

[0060] Optionally, generating a point cloud depth map of the target point cloud based on point cloud clusters of the target point cloud includes:

[0061] Calculate the centroid of each point cloud cluster in the target point cloud, and use the centroid of the point cloud cluster to represent the point cloud cluster;

[0062] The directions of the eigenvectors corresponding to the two larger eigenvalues ​​among the three directions of the centroid are taken as the two axis directions of the coordinate system plane;

[0063] Map the centroid of each point cloud cluster in the target point cloud to a planar grid in the coordinate system plane;

[0064] Calculate the depth information of each grid point, and obtain the point cloud depth map of the target point cloud based on the depth information;

[0065] Gaussian convolution is performed on the point cloud depth map to generate M Gaussian convolution point cloud depth maps.

[0066] Optionally, the point cloud depth map can be subjected to Gaussian convolution using the following formula:

[0067] L(x,y,σ)=G(x,y,σ)*I(x,y)

[0068]

[0069] Where L(x,y,σ) is the point cloud depth map after Gaussian convolution, I(x,y) is the point cloud depth map, σ=1.6*k, k represents the layer number of the point cloud depth map after Gaussian convolution, (m,n) is the pixel size, and (x,y) is the pixel coordinate.

[0070] Optionally, calculating the feature points to obtain feature point distribution feature descriptions of the source point cloud and the target point cloud includes:

[0071] Based on the feature points of the point cloud depth maps of the source point cloud and the target point cloud, find the spatial corresponding points. If there are multiple spatial corresponding points, retain the spatial corresponding points with the largest and smallest centroids at distances from the multiple spatial corresponding points.

[0072] The R neighborhood points of the corresponding points in the space are counted, and the local plane coordinate system of the principal component analysis algorithm is constructed based on the R neighborhood points. The eigenvector with the smallest eigenvalue is used as the point curvature description, the first eigenvector and the second eigenvector of the principal component analysis algorithm are used as the direction, and the center point of the point cloud cluster is used as the origin to form the local plane coordinate system.

[0073] The local plane coordinate system is divided into two parts with the same width, and then further divided into 45-degree segments. The minimum 200-degree plane normal vector and the center point of each region are calculated to form the six-dimensional information of the region.

[0074] The six-dimensional information of the region and the point curvature description are used as point feature information to generate a feature point distribution feature description.

[0075] Optionally, the similarity of the feature point distribution descriptions of the source point cloud and the target point cloud can be represented by the following formula:

[0076] s ij =α(n) i ·n j )+β(c i ·c j )

[0077] Wherein, the S ij This represents the similarity of the feature point distribution characteristics between the source point cloud and the target point cloud, where α and β are constants, and c... i and c j These are the six-dimensional information of the regions in the feature point distribution feature descriptions of the source point cloud and the target point cloud, respectively, n. i and n j These are the normalized normal vectors of the planes obtained by fitting the ground point clouds of the source point cloud and the target point cloud, respectively.

[0078] Optionally, determining the feature point matching relationship between the source point cloud and the target point cloud based on the feature points and the feature point distribution feature description includes:

[0079] Depth map feature points are extracted from the depth maps of the source point cloud and the target point cloud to generate 256-dimensional binary depth map features of the feature points of the source point cloud and the target point cloud. The 256-dimensional binary depth map features of the feature points of the source point cloud and the target point cloud are matched to obtain the matching result.

[0080] Based on the similarity of the feature point distribution features of the source point cloud and the target point cloud and the matching result, the feature point matching relationship between the source point cloud and the target point cloud is obtained.

[0081] Secondly, embodiments of the present invention provide an edge server, comprising:

[0082] A coarse registration module is used to perform coarse registration between a source point cloud and a target point cloud to obtain a coarsely registered point cloud of the target point cloud. The source point cloud is a point cloud obtained by measurement using a first measuring device, and the target point cloud is a point cloud obtained by measurement using a second measuring device. The first measuring device and the second measuring device have a common measurement area.

[0083] A precise registration module is used to precisely register the source point cloud and the coarse registration point cloud;

[0084] The process of coarsely registering the source point cloud and the target point cloud to obtain the coarsely registered point cloud of the target point cloud includes:

[0085] Obtain point cloud depth maps of the source point cloud and the target point cloud;

[0086] Feature points are obtained from the depth maps of the source point cloud and the target point cloud, and the feature points are calculated to obtain a feature point distribution feature description of the source point cloud and the target point cloud.

[0087] Based on the feature points and the feature point distribution feature description, determine the feature point matching relationship between the source point cloud and the target point cloud;

[0088] Based on the feature point matching relationship, the transformation matrix of the target point cloud is obtained;

[0089] Based on the transformation matrix, a coarse registration point cloud is obtained.

[0090] Optionally, performing point cloud correction on the preliminary transformed point cloud to obtain a coarse transformed point cloud includes:

[0091] Ground point cloud fitting is performed on the source point cloud and the preliminary transformed point cloud respectively to obtain the ground vectors of the source point cloud and the preliminary transformed point cloud;

[0092] The rotation angle of the preliminary transformed point cloud relative to the source point cloud is obtained based on the ground vector;

[0093] The initial converted point cloud is corrected based on the rotation angle to obtain the target point cloud.

[0094] Optionally, before the coarse registration module acquires the point cloud depth map of the target point cloud, it is also used to perform:

[0095] Based on the position information and angular extrinsic parameter calibration results of the source point cloud and the target point cloud, the target point cloud is initially transformed to obtain the initial transformed point cloud of the target point cloud;

[0096] Point cloud correction is performed on the preliminary transformed point cloud to obtain a coarse transformed point cloud corresponding to the target point cloud, thereby updating the target point cloud.

[0097] Optionally, ground point cloud fitting can be performed using the following formula:

[0098] Ax + By + Cz + D = 0

[0099] Where A, B, C, and D are constants, and x, y, and z are the location information of the point cloud data.

[0100] Optionally, the rotation angle can be calculated using the following formula:

[0101] θ = arccos(R, V)

[0102] Wherein, θ is the rotation angle, R is the ground vector of the initial converted point cloud, and V is the ground vector of the source point cloud.

[0103] Optionally, the preliminary transformed point cloud can be corrected using the following formula:

[0104] P ti =TP t1

[0105]

[0106] Among them, P ti For the target point cloud, P t1 Let T be the initial transformed point cloud, and T be the transformation matrix.

[0107] Optionally, obtaining the point cloud depth maps of the source point cloud and the target point cloud includes:

[0108] Point cloud clusters are aggregated for the source point cloud and the target point cloud respectively to obtain point cloud clusters of the source point cloud and the target point cloud, and the recognition box of each point cloud cluster is obtained.

[0109] The identification box of the point cloud cluster is voxelized to obtain multiple voxels. The centroid of the midpoint of each voxel is taken as the voxel center point. The N voxel center points closest to the voxel center point form N curvature surfaces. The surface normal vectors of the N curvature surfaces are calculated. The curvature information of the neighborhood surface of the voxel center point is obtained based on the surface normal vectors.

[0110] Based on the curvature information, the similarity between each point cloud cluster in the source point cloud and each point cloud cluster in the target point cloud is calculated. Based on the similarity, the most matching point cloud cluster and the most matching region in the target point cloud and the source point cloud are determined. The point cloud cluster with a larger number of voxels in the most matching region in the source point cloud or the target point cloud is replaced by a point cloud cluster with a smaller number of voxels.

[0111] A point cloud depth map of the source point cloud is generated based on the point cloud clusters of the source point cloud.

[0112] A point cloud depth map of the target point cloud is generated based on the point cloud clusters of the target point cloud.

[0113] Optional, N equals 3.

[0114] Optionally, obtaining the curvature information of the surface surrounding the voxel center point based on the surface normal vector includes:

[0115] The average value of the N surface normal vectors is used as the curvature information of the surface in the neighborhood of the voxel center point.

[0116] Optionally, calculating the similarity between each point cloud cluster in the source point cloud and each point cloud cluster in the target point cloud based on the curvature information includes:

[0117] The voxel space with a smaller number of voxels in the voxel space corresponding to the source point cloud and the target point cloud is selected as the voxel convolution kernel. It is compared with the voxel space with a larger number of voxels to obtain the voxel coefficient of the same corresponding space.

[0118] Based on the curvature information corresponding to the source point cloud and the target point cloud, calculate the curvature similarity coefficient between the source point cloud and the target point cloud;

[0119] The similarity between the point cloud clusters in the source point cloud and the point cloud clusters in the target point cloud is calculated based on the corresponding space occupancy voxel coefficient and the curvature similarity coefficient.

[0120] Optionally, the corresponding space-occupying voxel coefficient can be calculated using the following formula:

[0121]

[0122]

[0123] Where C0 is the voxel coefficient of the corresponding space occupancy, V min V represents the number of voxels in the voxel space of the point cloud with the smaller number of voxels. maxn is the number of voxels in the point cloud voxel space with a larger number of voxels, and n is the number of voxels with the same attribute in the voxel convolution kernel and the convolution region. The same attribute refers to the point cloud voxel space that is either point or no point.

[0124] Optionally, the curvature similarity coefficient can be calculated using the following formula:

[0125]

[0126]

[0127] Where c is the curvature similarity coefficient, curvature i For the curvature information corresponding to the source point cloud, curvature j For the curvature information corresponding to the target point cloud, α i α is an attribute of the voxels of the source point cloud. j The attribute of the voxels of the target point cloud is whether the voxel contains points or not.

[0128] Optionally, the similarity between point cloud clusters in the source point cloud and point cloud clusters in the target point cloud can be calculated using the following formula:

[0129] S=c*C0

[0130] Wherein, S is the similarity between the point cloud clusters in the source point cloud and the point cloud clusters in the target point cloud, c is the curvature similarity coefficient, and C0 is the voxel coefficient of the same corresponding space.

[0131] Optionally, generating a point cloud depth map of the source point cloud based on the point cloud clusters of the source point cloud includes:

[0132] Calculate the centroid of each point cloud cluster in the source point cloud, and use the centroid of the point cloud cluster to represent the point cloud cluster;

[0133] The directions of the eigenvectors corresponding to the two larger eigenvalues ​​among the three directions of the centroid are taken as the two axis directions of the coordinate system plane;

[0134] Map the centroid of each point cloud cluster in the source point cloud to a planar grid in the coordinate system plane;

[0135] Calculate the depth information of each grid point, and obtain the point cloud depth map of the source point cloud based on the depth information.

[0136] Optionally, generating a point cloud depth map of the target point cloud based on point cloud clusters of the target point cloud includes:

[0137] Calculate the centroid of each point cloud cluster in the target point cloud, and use the centroid of the point cloud cluster to represent the point cloud cluster;

[0138] The directions of the eigenvectors corresponding to the two larger eigenvalues ​​among the three directions of the centroid are taken as the two axis directions of the coordinate system plane;

[0139] Map the centroid of each point cloud cluster in the target point cloud to a planar grid in the coordinate system plane;

[0140] Calculate the depth information of each grid point, and obtain the point cloud depth map of the target point cloud based on the depth information;

[0141] Gaussian convolution is performed on the point cloud depth map to generate M Gaussian convolution point cloud depth maps.

[0142] Optionally, the point cloud depth map can be subjected to Gaussian convolution using the following formula:

[0143] L(x,y,σ)=G(x,y,σ)*I(x,y)

[0144]

[0145] Where L(x,y,σ) is the point cloud depth map after Gaussian convolution, I(x,y) is the point cloud depth map, σ=1.6*k, k represents the layer number of the point cloud depth map after Gaussian convolution, (m,n) is the pixel size, and (x,y) is the pixel coordinate.

[0146] Optionally, calculating the feature points to obtain feature point distribution feature descriptions of the source point cloud and the target point cloud includes:

[0147] Based on the feature points of the point cloud depth maps of the source point cloud and the target point cloud, find the spatial corresponding points. If there are multiple spatial corresponding points, retain the spatial corresponding points with the largest and smallest centroids at distances from the multiple spatial corresponding points.

[0148] The R neighborhood points of the corresponding points in the space are counted, and the local plane coordinate system of the principal component analysis algorithm is constructed based on the R neighborhood points. The eigenvector with the smallest eigenvalue is used as the point curvature description, the first eigenvector and the second eigenvector of the principal component analysis algorithm are used as the direction, and the center point of the point cloud cluster is used as the origin to form the local plane coordinate system.

[0149] The local plane coordinate system is divided into two parts with the same width, and then further divided into 45-degree segments. The minimum 200-degree plane normal vector and the center point of each region are calculated to form the six-dimensional information of the region.

[0150] The six-dimensional information of the region and the point curvature description are used as point feature information to generate a feature point distribution feature description.

[0151] Optionally, the similarity of the feature point distribution descriptions of the source point cloud and the target point cloud can be represented by the following formula:

[0152] s ij =α(n) i ·n j )+β(c i ·c j )

[0153] Wherein, the S ij This represents the similarity of the feature point distribution characteristics between the source point cloud and the target point cloud, where α and β are constants, and c... i and c j These are the six-dimensional information of the regions in the feature point distribution feature descriptions of the source point cloud and the target point cloud, respectively, n. i and n j These are the normalized normal vectors of the planes obtained by fitting the ground point clouds of the source point cloud and the target point cloud, respectively.

[0154] Optionally, determining the feature point matching relationship between the source point cloud and the target point cloud based on the feature points and the feature point distribution feature description includes:

[0155] Depth map feature points are extracted from the depth maps of the source point cloud and the target point cloud to generate 256-dimensional binary depth map features of the feature points of the source point cloud and the target point cloud. The 256-dimensional binary depth map features of the feature points of the source point cloud and the target point cloud are matched to obtain the matching result.

[0156] Based on the similarity of the feature point distribution features of the source point cloud and the target point cloud and the matching result, the feature point matching relationship between the source point cloud and the target point cloud is obtained.

[0157] Thirdly, embodiments of the present invention provide an edge server, including: a processor, a memory, and a program stored in the memory and executable on the processor, wherein when the program is executed by the processor, it implements the steps of the point cloud registration method as described in the first aspect above.

[0158] Fourthly, embodiments of the present invention provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the point cloud registration method described in the first aspect above.

[0159] In this embodiment of the invention, when performing coarse registration of the source point cloud and the target point cloud, a method combining depth map features and feature point distribution features is used for coarse registration. Since converting the point cloud into a depth map maps the point cloud data to a planar coordinate system, the density of the dense point cloud becomes sparse. When the density difference between the two point clouds to be registered is large, it can also provide a better registration effect. Attached Figure Description

[0160] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0161] Figure 1 This is a schematic diagram of a point cloud registration system according to an embodiment of the present invention;

[0162] Figure 2 This is a flowchart illustrating the point cloud registration method according to an embodiment of the present invention;

[0163] Figure 3 This is a schematic diagram of the method for coarse registration of the source point cloud and the target point cloud according to an embodiment of the present invention;

[0164] Figure 4 This is a schematic diagram of voxelization processing of the recognition box of point cloud clusters according to an embodiment of the present invention;

[0165] Figure 5 This is a schematic diagram illustrating the generation of point cloud curvature information according to an embodiment of the present invention;

[0166] Figure 6 This is a schematic diagram illustrating spatial convolution using voxel convolution kernels in an embodiment of the present invention;

[0167] Figure 7 and Figure 8 This is a schematic diagram of a point cloud depth map generation method according to an embodiment of the present invention;

[0168] Figure 9 This is a schematic diagram illustrating the feature point distribution characteristics of an embodiment of the present invention;

[0169] Figure 10 This is a schematic diagram of the structure of an edge server according to an embodiment of the present invention;

[0170] Figure 11 This is a schematic diagram of the structure of an edge server according to another embodiment of the present invention. Detailed Implementation

[0171] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0172] Please refer to Figure 1 , Figure 1 This is a schematic diagram of a point cloud registration system according to an embodiment of the present invention. The system includes multiple lidar units installed on the roadside and a roadside edge server connected to the lidar units. The maximum sensing range of each lidar unit can be, for example, 200m, and they are installed on the roadside at intervals of 110m. The lidar units feed back the sensed road point cloud data to the roadside edge server. Adjacent lidar units share a common measurement area. By stitching together the shared point cloud information of different lidar units in the common measurement area, the coordinate system transformation relationship of different lidar units can be obtained, enabling cross-device sensing and tracking. However, due to the varying distances between the lidar units, the point cloud density is higher for lidar units closer to the common measurement area, while the point cloud density is lower for lidar units farther away. This can easily lead to poor stitching results during point cloud feature extraction.

[0173] To solve the above problems, please refer to... Figure 2 This invention provides a point cloud registration method, comprising:

[0174] Step 1: Perform coarse registration on the source point cloud and the target point cloud to obtain a coarsely registered point cloud of the target point cloud. The source point cloud is a point cloud obtained by measuring with a first measuring device, and the target point cloud is a point cloud obtained by measuring with a second measuring device. The first measuring device and the second measuring device have a common measurement area.

[0175] Step 2: Perform precise registration between the source point cloud and the coarse registration point cloud;

[0176] Please refer to the following: Figure 3 In step 1, the coarse registration of the source point cloud and the target point cloud to obtain the coarsely registered point cloud of the target point cloud includes:

[0177] Step 11: Obtain the point cloud depth maps of the source point cloud and the target point cloud;

[0178] Step 12: Obtain feature points from the depth maps of the source point cloud and the target point cloud, calculate the feature points, and obtain a feature point distribution description of the source point cloud and the target point cloud;

[0179] Step 13: Determine the feature point matching relationship between the source point cloud and the target point cloud based on the feature points and the feature point distribution feature description;

[0180] Step 14: Obtain the transformation matrix of the target point cloud based on the feature point matching relationship;

[0181] Step 15: Obtain the coarse registration point cloud based on the transformation matrix.

[0182] In this embodiment of the invention, when performing coarse registration of the source point cloud and the target point cloud, a method combining depth map features and feature point distribution features is used for coarse registration. Since converting the point cloud into a depth map maps the point cloud data to a planar coordinate system, the density of the dense point cloud becomes sparse. When the density difference between the two point clouds to be registered is large, it can also provide a better registration effect.

[0183] In this embodiment of the invention, optionally, before obtaining the point cloud depth map of the target point cloud in step 11, the method further includes:

[0184] Step 10A: Based on the position information and angle extrinsic parameter calibration results of the source point cloud and the target point cloud, perform a preliminary transformation on the target point cloud to obtain a preliminary transformed point cloud of the target point cloud;

[0185] The location information may be, for example, GPS (Global Positioning System) location information.

[0186] Step 10B: Perform point cloud correction on the preliminary transformed point cloud to obtain the coarse transformed point cloud corresponding to the target point cloud and update the target point cloud.

[0187] In this embodiment of the invention, optionally, step 10B, performing point cloud correction on the preliminary converted point cloud to obtain a coarse converted point cloud, includes:

[0188] Step 10B1: Perform ground point cloud fitting on the source point cloud and the preliminary converted point cloud respectively to obtain the ground vectors of the source point cloud and the preliminary converted point cloud;

[0189] In this embodiment of the invention, optionally, the ransac (RANDOM SAmple Consensus) algorithm can be used to process the source point cloud P. s With P t1 Ground point cloud fitting can be performed using the following formula:

[0190] Ax+By+Cz+D=0………………(1)

[0191] Where A, B, C, and D are constants, and x, y, and z are the location information of the point cloud data.

[0192] Step 10B 2: Obtain the rotation angle of the preliminary transformed point cloud relative to the source point cloud based on the ground vector;

[0193] In this embodiment of the invention, optionally, the rotation angle is calculated using the following formula:

[0194] θ=arccos(R,V)…………(2)

[0195] Wherein, θ is the rotation angle, R is the ground vector of the initial converted point cloud, and V is the ground vector of the source point cloud.

[0196] Step 10B 3: Perform point cloud correction on the preliminary converted point cloud according to the rotation angle to obtain the target point cloud.

[0197] In this embodiment of the invention, optionally, the preliminary converted point cloud is corrected using the following formula:

[0198] P ti =TP t1 ………………(3)

[0199]

[0200] Among them, P ti For the target point cloud, P t1 Let T be the initial transformed point cloud, and T be the transformation matrix.

[0201] In this embodiment of the invention, optionally, step 11, obtaining the point cloud depth maps of the source point cloud and the target point cloud, includes:

[0202] Step 111: Perform point cloud cluster aggregation on the source point cloud and the target point cloud respectively to obtain the point cloud clusters of the source point cloud and the target point cloud, and obtain the recognition box of each point cloud cluster.

[0203] In this embodiment of the invention, optionally, the source point cloud and the target point cloud can be clustered based on the DBSCAN (Density-Based Spatial Clustering of Applications with Noise) algorithm, and the centroid, length, width and height of each point cloud cluster can be calculated to form the identification box of the point cloud cluster.

[0204] Step 112: Perform voxelization on the recognition box of the point cloud cluster to obtain multiple voxels, and take the centroid of the midpoint of each voxel as the voxel center point. The N voxel center points closest to the voxel center point form N curvature surfaces, and calculate the surface normal vector of the N curvature surfaces. Obtain the curvature information of the neighborhood surface of the voxel center point based on the surface normal vector.

[0205] In this embodiment of the invention, the recognition box of the point cloud cluster can be voxelized to a size of 0.1m*0.1m, and the obtained voxels are as follows: Figure 4 As shown.

[0206] In this embodiment of the invention, N is optionally equal to 3. Of course, N can also be other values. When N equals 3, the obtained curvature surface is as follows: Figure 5 As shown.

[0207] Optionally, obtaining the curvature information of the neighborhood surface of the voxel center point based on the surface normal vector includes: taking the average value of the N surface normal vectors as the curvature information of the neighborhood surface of the voxel center point.

[0208] Step 113: Based on the curvature information, calculate the similarity between each point cloud cluster in the source point cloud and each point cloud cluster in the target point cloud. Based on the similarity, determine the most matching point cloud cluster and the most matching region in the target point cloud and the source point cloud. Replace the point cloud cluster with a smaller number of voxels in the most matching region of the source point cloud or the target point cloud with a point cloud cluster with a smaller number of voxels.

[0209] In this embodiment of the invention, the similarity between the point cloud clusters in the source point cloud and each point cloud cluster in the target point cloud is calculated to find the most matching transformed point cloud cluster in each source point cloud cluster.

[0210] In this embodiment of the invention, optionally, calculating the similarity between each point cloud cluster in the source point cloud and each point cloud cluster in the target point cloud includes:

[0211] 1) Please refer to Figure 6 The voxel space with a smaller number of voxels in the voxel space corresponding to the source point cloud and the target point cloud is selected as the voxel convolution kernel. It is compared with the voxel space with a larger number of voxels to obtain the voxel coefficient of the same corresponding space.

[0212] Optionally, the corresponding space-occupying voxel coefficient can be calculated using the following formula:

[0213]

[0214]

[0215] Where C0 is the voxel coefficient of the corresponding space occupancy, V min V represents the number of voxels in the voxel space of the point cloud with the smaller number of voxels. max n is the number of voxels in the point cloud voxel space with a larger number of voxels, and n is the number of voxels with the same attribute in the voxel convolution kernel and the convolution region. The same attribute refers to the point cloud voxel space that is either point or no point.

[0216] 2) Calculate the curvature similarity coefficient between the source point cloud and the target point cloud based on the curvature information corresponding to the source point cloud and the target point cloud;

[0217] Optionally, the curvature similarity coefficient between the source point cloud and the target point cloud can be calculated using the following formula:

[0218]

[0219]

[0220] Where c is the curvature similarity coefficient, curvature i For the curvature information corresponding to the source point cloud, curvature j For the curvature information corresponding to the target point cloud, α i α is an attribute of the voxels of the source point cloud. j The attribute of the voxels of the target point cloud is whether the voxel contains points or not.

[0221] 3) Calculate the similarity between the point cloud clusters in the source point cloud and the point cloud clusters in the target point cloud based on the same corresponding space occupancy voxel coefficient and the curvature similarity coefficient.

[0222] Optionally, the similarity between point cloud clusters in the source point cloud and point cloud clusters in the target point cloud can be calculated using the following formula:

[0223] S=c*C0………………(9)

[0224] Wherein, S is the similarity between the point cloud clusters in the source point cloud and the point cloud clusters in the target point cloud, c is the curvature similarity coefficient, and C0 is the voxel coefficient of the same corresponding space.

[0225] Step 114: Generate a point cloud depth map of the source point cloud based on the point cloud clusters of the source point cloud;

[0226] Optionally, generating a point cloud depth map of the source point cloud based on the point cloud clusters of the source point cloud includes:

[0227] 1) Calculate the centroid of each point cloud cluster in the source point cloud, and use the centroid of the point cloud cluster to represent the point cloud cluster;

[0228] In this embodiment of the invention, assuming that the point cloud cluster includes 5 point cloud data, the eigenvalues ​​of the 5 point cloud data in each eigenvector direction are added together and averaged to obtain the eigenvalues ​​of the centroid of the point cloud cluster in each eigenvector direction.

[0229] 2) Take the directions of the eigenvectors corresponding to the two larger eigenvalues ​​among the three directions of the centroid as the two axis directions of the coordinate system plane;

[0230] Please refer to Figure 7 , Figure 7 This is a schematic diagram of the eigenvectors in three directions of the centroid of a point cloud cluster. Figure 7 As can be seen, the eigenvalues ​​of the centroid are larger in the u and v directions and smaller in the w direction.

[0231] 3) Map the centroid of each point cloud cluster in the source point cloud to a planar grid in the coordinate system plane;

[0232] Please refer to Figure 8 , Figure 8 It is a schematic diagram of mapping the centroid of each point cloud cluster onto a planar grid in the coordinate system plane.

[0233] In this embodiment of the invention, optionally, the planar grid can be a 64*64 planar grid, thereby forming a 64*64 point cloud depth map. Of course, the planar grid is not limited to 64*64.

[0234] 4) Calculate the depth information of each grid point, and obtain the point cloud depth map of the source point cloud based on the depth information.

[0235] In this embodiment of the invention, optionally, after obtaining the point cloud depth map of the source point cloud based on the depth information, Gaussian filtering can be applied to the point cloud depth map to compensate for the loss of feature information caused by the reduction of point cloud density.

[0236] Step 115: Generate a point cloud depth map of the target point cloud based on the point cloud clusters of the target point cloud.

[0237] In this embodiment of the invention, in order to simulate the blurring of the depth map caused by visual distance, a Gaussian pyramid is used to generate the point cloud depth map of the coarsely transformed point cloud.

[0238] Optionally, generating a point cloud depth map of the target point cloud based on point cloud clusters of the target point cloud includes:

[0239] 1) Calculate the centroid of each point cloud cluster in the target point cloud, and use the centroid of the point cloud cluster to represent the point cloud cluster;

[0240] In this embodiment of the invention, assuming that the point cloud cluster includes 5 point cloud data, the eigenvalues ​​of the 5 point cloud data in each eigenvector direction are added together and averaged to obtain the eigenvalues ​​of the centroid of the point cloud cluster in each eigenvector direction.

[0241] 2) Take the directions of the eigenvectors corresponding to the two larger eigenvalues ​​among the three directions of the centroid as the two axis directions of the coordinate system plane;

[0242] 3) Map the centroid of each point cloud cluster in the target point cloud to a planar grid in the coordinate system plane;

[0243] In this embodiment of the invention, optionally, the planar grid can be a 256*256 planar grid, thereby forming a 256*256 point cloud depth map. Through Gaussian pyramid processing, three images of size 64*64 are retained. Of course, the planar grid is not limited to 256*256.

[0244] 4) Calculate the depth information of each grid point, and obtain the point cloud depth map of the target point cloud based on the depth information;

[0245] In this embodiment of the invention, optionally, after obtaining the point cloud depth map of the source point cloud based on the depth information, Gaussian filtering can be applied to the point cloud depth map to compensate for the loss of feature information caused by the reduction of point cloud density.

[0246] 5) Perform Gaussian convolution on the point cloud depth map to generate M Gaussian convolution point cloud depth maps.

[0247] Optionally, the point cloud depth map can be subjected to Gaussian convolution using the following formula:

[0248] L(x,y,σ)=G(x,y,σ)*I(x,y)…………(10)

[0249]

[0250] Where L(x,y,σ) is the point cloud depth map after Gaussian convolution, I(x,y) is the point cloud depth map, σ=1.6*k, k represents the layer number of the point cloud depth map after Gaussian convolution, (m,n) is the pixel size, and (x,y) is the pixel coordinate.

[0251] In this embodiment of the invention, M can optionally be 3, that is, three Gaussian convolution pyramid images (point cloud depth maps after Gaussian convolution) are generated. In other words, the Gaussian convolution pyramid has three layers.

[0252] In this embodiment of the invention, optionally, in step 14, the fast corner detection operator can be used to obtain feature points in the (M) point cloud depth maps of the source point cloud and the target point cloud, and find the spatial corresponding points (three-dimensional spatial points) of the feature points. If there are multiple corresponding spatial points, they are recorded as a set of feature points.

[0253] After the point cloud depth map is processed by Gaussian filtering, the curvature information of the point cloud is smoothed, resulting in feature loss. Therefore, in this embodiment of the invention, in addition to collecting depth map features, feature point distribution feature description calculation is also performed.

[0254] In this embodiment of the invention, optionally, step 12, calculating the feature points to obtain the feature point distribution feature descriptions of the source point cloud and the target point cloud, includes:

[0255] Step 121: Based on the feature points of the point cloud depth maps of the source point cloud and the target point cloud, find the spatial corresponding points. If there are multiple spatial corresponding points, retain the spatial corresponding points with the largest and smallest centroids at distances from the multiple spatial corresponding points.

[0256] Step 122: Count the R neighborhood points of the corresponding points in the space, and construct the local plane coordinate system of the Principal Component Analysis (PCA) algorithm based on the R neighborhood points. The eigenvector with the smallest eigenvalue is used as the point curvature description, the first and second eigenvectors of the PCA algorithm are used as the direction, and the center point of the point cloud cluster is used as the origin to form the local plane coordinate system.

[0257] Step 123: Divide the local plane coordinate system into two parts with the same width, and then divide it into 45-degree segments. Calculate the minimum 200-degree plane normal vector and center point of each region to form the six-dimensional information of the region.

[0258] Step 124: Use the six-dimensional information of the region and the point curvature description as point feature information to generate a feature point distribution feature description.

[0259] Please refer to Figure 9 , Figure 9 This is a schematic diagram illustrating the feature point distribution characteristics of an embodiment of the present invention.

[0260] Optionally, the similarity of the feature point distribution descriptions of the source point cloud and the target point cloud can be represented by the following formula:

[0261] s ij =α(n) i ·n j )+β(c i ·c j )………………(12)

[0262] Wherein, the S ij This represents the similarity of the feature point distribution characteristics between the source point cloud and the target point cloud, where α and β are constants, and c... i and c j These are the six-dimensional information of the regions in the feature point distribution feature descriptions of the source point cloud and the target point cloud, respectively, n. i and n j These are the normalized normal vectors of the planes obtained by fitting the ground point clouds of the source point cloud and the target point cloud, respectively.

[0263] Optionally, in step 13, determining the feature point matching relationship between the source point cloud and the target point cloud based on the feature points and the feature point distribution feature description includes:

[0264] Step 131: Extract depth map feature points from the source point cloud and the target point cloud using the FAST algorithm, generate 256-dimensional binary depth map features of the source point cloud and the target point cloud feature points, and match the 256-dimensional binary depth map features of the source point cloud and the target point cloud feature points to obtain the matching result;

[0265] Specifically, based on the feature points of the point cloud depth map of the source point cloud, a 256-dimensional binary depth map feature of the source point cloud is generated.

[0266] Based on the feature points of the M point cloud depth maps of the target point cloud, generate M 256-dimensional binary depth map features of the target point cloud.

[0267] Using Hamming distance, the 256-dimensional binary depth map features of the source point cloud are matched with M sets of 256-dimensional binary depth map features of the coarsely transformed point cloud, and the best matching 256-dimensional binary depth map feature is found from the M sets of 256-dimensional binary depth map features.

[0268] Step 132: Based on the similarity of the feature point distribution features of the source point cloud and the target point cloud and the matching result, obtain the feature point matching relationship between the source point cloud and the target point cloud.

[0269] In this embodiment of the invention, in order to represent the original point cloud distribution features as much as possible and achieve the goals of resisting point density changes and uniqueness in the feature point distribution feature description, a method combining depth map features and feature point distribution features is adopted to obtain the feature point matching relationship between the source point cloud and the target point cloud.

[0270] In this embodiment of the invention, the ICP (Iterative Closest Point) algorithm can be used to accurately register the source point cloud and the coarse registration point cloud. Of course, it is also possible to use other existing algorithms to accurately register the source point cloud and the coarse registration point cloud.

[0271] Please refer to Figure 10 This invention also provides an edge server 100, comprising:

[0272] The coarse registration module 101 is used to perform coarse registration on the source point cloud and the target point cloud to obtain the coarse registration point cloud of the target point cloud. The source point cloud is a point cloud obtained by measuring with a first measuring device, and the target point cloud is a point cloud obtained by measuring with a second measuring device. The first measuring device and the second measuring device have a common measurement area.

[0273] The precise registration module 102 is used to perform precise registration between the source point cloud and the coarse registration point cloud;

[0274] The process of coarsely registering the source point cloud and the target point cloud to obtain the coarsely registered point cloud of the target point cloud includes:

[0275] Obtain point cloud depth maps of the source point cloud and the target point cloud;

[0276] Feature points are obtained from the depth maps of the source point cloud and the target point cloud, and the feature points are calculated to obtain a feature point distribution feature description of the source point cloud and the target point cloud.

[0277] Based on the feature points and the feature point distribution feature description, determine the feature point matching relationship between the source point cloud and the target point cloud;

[0278] Based on the feature point matching relationship, the transformation matrix of the target point cloud is obtained;

[0279] Based on the transformation matrix, a coarse registration point cloud is obtained.

[0280] Optionally, the coarse registration module 101 is configured to perform the following before acquiring the point cloud depth map of the target point cloud:

[0281] Based on the position information and angular extrinsic parameter calibration results of the source point cloud and the target point cloud, the target point cloud is initially transformed to obtain the initial transformed point cloud of the target point cloud;

[0282] Point cloud correction is performed on the preliminary transformed point cloud to obtain a coarse transformed point cloud corresponding to the target point cloud, thereby updating the target point cloud.

[0283] Optionally, performing point cloud correction on the preliminary transformed point cloud to obtain a coarse transformed point cloud includes:

[0284] Ground point cloud fitting is performed on the source point cloud and the preliminary transformed point cloud respectively to obtain the ground vectors of the source point cloud and the preliminary transformed point cloud;

[0285] The rotation angle of the preliminary transformed point cloud relative to the source point cloud is obtained based on the ground vector;

[0286] The initial converted point cloud is corrected based on the rotation angle to obtain the target point cloud.

[0287] Optionally, ground point cloud fitting can be performed using the following formula:

[0288] Ax + By + Cz + D = 0

[0289] Where A, B, C, and D are constants, and x, y, and z are the location information of the point cloud data.

[0290] Optionally, the rotation angle can be calculated using the following formula:

[0291] θ = arccos(R, V)

[0292] Wherein, θ is the rotation angle, R is the ground vector of the initial converted point cloud, and V is the ground vector of the source point cloud.

[0293] Optionally, the preliminary transformed point cloud can be corrected using the following formula:

[0294] P ti =TP t1

[0295]

[0296] Among them, P ti For the target point cloud, P t1 Let T be the initial transformed point cloud, and T be the transformation matrix.

[0297] Optionally, obtaining the point cloud depth maps of the source point cloud and the target point cloud includes:

[0298] Point cloud clusters are aggregated for the source point cloud and the target point cloud respectively to obtain point cloud clusters of the source point cloud and the target point cloud, and the recognition box of each point cloud cluster is obtained.

[0299] The identification box of the point cloud cluster is voxelized to obtain multiple voxels. The centroid of the midpoint of each voxel is taken as the voxel center point. The N voxel center points closest to the voxel center point form N curvature surfaces. The surface normal vectors of the N curvature surfaces are calculated. The curvature information of the neighborhood surface of the voxel center point is obtained based on the surface normal vectors.

[0300] Based on the curvature information, the similarity between each point cloud cluster in the source point cloud and each point cloud cluster in the target point cloud is calculated. Based on the similarity, the most matching point cloud cluster and the most matching region in the target point cloud and the source point cloud are determined. The point cloud cluster with a larger number of voxels in the most matching region in the source point cloud or the target point cloud is replaced by a point cloud cluster with a smaller number of voxels.

[0301] A point cloud depth map of the source point cloud is generated based on the point cloud clusters of the source point cloud.

[0302] A point cloud depth map of the target point cloud is generated based on the point cloud clusters of the target point cloud.

[0303] Optional, N equals 3.

[0304] Optionally, obtaining the curvature information of the surface surrounding the voxel center point based on the surface normal vector includes:

[0305] The average value of the N surface normal vectors is used as the curvature information of the surface in the neighborhood of the voxel center point.

[0306] Optionally, calculating the similarity between each point cloud cluster in the source point cloud and each point cloud cluster in the target point cloud based on the curvature information includes:

[0307] The voxel space with a smaller number of voxels in the voxel space corresponding to the source point cloud and the target point cloud is selected as the voxel convolution kernel. It is compared with the voxel space with a larger number of voxels to obtain the voxel coefficient of the same corresponding space.

[0308] Based on the curvature information corresponding to the source point cloud and the target point cloud, calculate the curvature similarity coefficient between the source point cloud and the target point cloud;

[0309] The similarity between the point cloud clusters in the source point cloud and the point cloud clusters in the target point cloud is calculated based on the corresponding space occupancy voxel coefficient and the curvature similarity coefficient.

[0310] Optionally, the corresponding space-occupying voxel coefficient can be calculated using the following formula:

[0311]

[0312]

[0313] Where C0 is the voxel coefficient of the corresponding space occupancy, V min V represents the number of voxels in the voxel space of the point cloud with the smaller number of voxels. max n is the number of voxels in the point cloud voxel space with a larger number of voxels, and n is the number of voxels with the same attribute in the voxel convolution kernel and the convolution region. The same attribute refers to the point cloud voxel space that is either point or no point.

[0314] Optionally, the curvature similarity coefficient can be calculated using the following formula:

[0315]

[0316]

[0317] Where c is the curvature similarity coefficient, curvature i For the curvature information corresponding to the source point cloud, curvature j For the curvature information corresponding to the target point cloud, α i α is an attribute of the voxels of the source point cloud. j The attribute of the voxels of the target point cloud is whether the voxel contains points or not.

[0318] Optionally, the similarity between point cloud clusters in the source point cloud and point cloud clusters in the target point cloud can be calculated using the following formula:

[0319] S=c*C0

[0320] Wherein, S is the similarity between the point cloud clusters in the source point cloud and the point cloud clusters in the target point cloud, c is the curvature similarity coefficient, and C0 is the voxel coefficient of the same corresponding space.

[0321] Optionally, generating a point cloud depth map of the source point cloud based on the point cloud clusters of the source point cloud includes:

[0322] Calculate the centroid of each point cloud cluster in the source point cloud, and use the centroid of the point cloud cluster to represent the point cloud cluster;

[0323] The directions of the eigenvectors corresponding to the two larger eigenvalues ​​among the three directions of the centroid are taken as the two axis directions of the coordinate system plane;

[0324] Map the centroid of each point cloud cluster in the source point cloud to a planar grid in the coordinate system plane;

[0325] Calculate the depth information of each grid point, and obtain the point cloud depth map of the source point cloud based on the depth information.

[0326] Optionally, generating a point cloud depth map of the target point cloud based on point cloud clusters of the target point cloud includes:

[0327] Calculate the centroid of each point cloud cluster in the target point cloud, and use the centroid of the point cloud cluster to represent the point cloud cluster;

[0328] The directions of the eigenvectors corresponding to the two larger eigenvalues ​​among the three directions of the centroid are taken as the two axis directions of the coordinate system plane;

[0329] Map the centroid of each point cloud cluster in the target point cloud to a planar grid in the coordinate system plane;

[0330] Calculate the depth information of each grid point, and obtain the point cloud depth map of the target point cloud based on the depth information;

[0331] Gaussian convolution is performed on the point cloud depth map to generate M Gaussian convolution point cloud depth maps.

[0332] Optionally, the point cloud depth map can be subjected to Gaussian convolution using the following formula:

[0333] L(x,y,σ)=G(x,y,σ)*I(x,y)

[0334]

[0335] Where L(x,y,σ) is the point cloud depth map after Gaussian convolution, I(x,y) is the point cloud depth map, σ=1.6*k, k represents the layer number of the point cloud depth map after Gaussian convolution, (m,n) is the pixel size, and (x,y) is the pixel coordinate.

[0336] Optionally, calculating the feature points to obtain feature point distribution feature descriptions of the source point cloud and the target point cloud includes:

[0337] Based on the feature points of the point cloud depth maps of the source point cloud and the target point cloud, find the spatial corresponding points. If there are multiple spatial corresponding points, retain the spatial corresponding points with the largest and smallest centroids at distances from the multiple spatial corresponding points.

[0338] The R neighborhood points of the corresponding points in the space are counted, and the local plane coordinate system of the principal component analysis algorithm is constructed based on the R neighborhood points. The eigenvector with the smallest eigenvalue is used as the point curvature description, the first eigenvector and the second eigenvector of the principal component analysis algorithm are used as the direction, and the center point of the point cloud cluster is used as the origin to form the local plane coordinate system.

[0339] The local plane coordinate system is divided into two parts with the same width, and then further divided into 45-degree segments. The minimum 200-degree plane normal vector and the center point of each region are calculated to form the six-dimensional information of the region.

[0340] The six-dimensional information of the region and the point curvature description are used as point feature information to generate a feature point distribution feature description.

[0341] Optionally, the similarity of the feature point distribution descriptions of the source point cloud and the target point cloud can be represented by the following formula:

[0342] s ij =α(n) i ·n j )+β(c i ·c j )

[0343] Wherein, the S ij This represents the similarity of the feature point distribution characteristics between the source point cloud and the target point cloud, where α and β are constants, and c... i and c j These are the six-dimensional information of the regions in the feature point distribution feature descriptions of the source point cloud and the target point cloud, respectively, n. i and n j These are the normalized normal vectors of the planes obtained by fitting the ground point clouds of the source point cloud and the target point cloud, respectively.

[0344] Optionally, determining the feature point matching relationship between the source point cloud and the target point cloud based on the feature points and the feature point distribution feature description includes:

[0345] For the point cloud depth maps of the source point cloud and the target point cloud, depth map feature points are extracted to generate 256-dimensional binary depth map features of the feature points of the source point cloud and the target point cloud. The 256-dimensional binary depth map features of the feature points of the source point cloud and the target point cloud are matched to obtain the matching result.

[0346] Based on the similarity of the feature point distribution features of the source point cloud and the target point cloud and the matching result, the feature point matching relationship between the source point cloud and the target point cloud is obtained.

[0347] Please refer to Figure 11 The present invention also provides an edge server 110, including a processor 111, a memory 112, and a computer program stored in the memory 112 and executable on the processor 111. When the computer program is executed by the processor 111, it implements the various processes of the above-described point cloud registration method embodiments and achieves the same technical effect. To avoid repetition, it will not be described again here.

[0348] This invention also provides a computer-readable storage medium storing a computer program. When executed by a processor, the computer program implements the various processes of the point cloud registration method embodiments described above and achieves the same technical effects. To avoid repetition, these will not be described again here. The computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc.

[0349] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0350] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0351] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of the present invention.

Claims

1. A point cloud registration method, characterized in that, include: Coarse registration is performed on the source point cloud and the target point cloud to obtain a coarsely registered point cloud of the target point cloud. The source point cloud is a point cloud obtained by measuring with a first measuring device, and the target point cloud is a point cloud obtained by measuring with a second measuring device. The first measuring device and the second measuring device have a common measurement area. Perform precise registration between the source point cloud and the coarse registration point cloud; The process of coarsely registering the source point cloud and the target point cloud to obtain the coarsely registered point cloud of the target point cloud includes: Obtain point cloud depth maps of the source point cloud and the target point cloud; Feature points are obtained from the depth maps of the source point cloud and the target point cloud, and the feature points are calculated to obtain a feature point distribution feature description of the source point cloud and the target point cloud. Based on the feature points and the feature point distribution feature description, determine the feature point matching relationship between the source point cloud and the target point cloud; Based on the feature point matching relationship, the transformation matrix of the target point cloud is obtained; Based on the transformation matrix, a coarsely matched point cloud is obtained; The calculation of the feature points to obtain the feature point distribution feature descriptions of the source point cloud and the target point cloud includes: Based on the feature points of the point cloud depth maps of the source point cloud and the target point cloud, find the spatial corresponding points. If there are multiple spatial corresponding points, retain the spatial corresponding points with the largest and smallest centroids at distances from the multiple spatial corresponding points. The R neighborhood points of the corresponding points in the space are counted, and the local plane coordinate system of the principal component analysis algorithm is constructed based on the R neighborhood points. The eigenvector with the smallest eigenvalue is used as the point curvature description, the first eigenvector and the second eigenvector of the principal component analysis algorithm are used as the direction, and the centroid of the point cloud cluster is used as the origin to form the local plane coordinate system. The local plane coordinate system is divided into two parts with the same width, and then further divided into 45-degree segments. The minimum 200-degree plane normal vector and the center point of each region are calculated to form the six-dimensional information of the region. The six-dimensional information of the region and the point curvature description are used as point feature information to generate a feature point distribution feature description.

2. The method as described in claim 1, characterized in that, Before obtaining the point cloud depth map of the target point cloud, the following steps are also included: Based on the position information and angular extrinsic parameter calibration results of the source point cloud and the target point cloud, the target point cloud is initially transformed to obtain the initial transformed point cloud of the target point cloud; Point cloud correction is performed on the preliminary transformed point cloud to obtain a coarse transformed point cloud corresponding to the target point cloud, thereby updating the target point cloud.

3. The method as described in claim 2, characterized in that, Performing point cloud correction on the preliminary transformed point cloud to obtain the coarse transformed point cloud corresponding to the target point cloud includes: Ground point cloud fitting is performed on the source point cloud and the preliminary transformed point cloud respectively to obtain the ground vectors of the source point cloud and the preliminary transformed point cloud; The rotation angle of the preliminary transformed point cloud relative to the source point cloud is obtained based on the ground vector; The point cloud is corrected based on the rotation angle to obtain the coarse converted point cloud.

4. The method as described in claim 3, characterized in that, Ground point cloud fitting is performed using the following formula: ； Where A, B, C, and D are constants, and x, y, and z are the location information of the point cloud data.

5. The method as described in claim 4, characterized in that, The rotation angle is calculated using the following formula: ； in, Let R be the rotation angle, R be the ground vector of the initial converted point cloud, and V be the ground vector of the source point cloud.

6. The method as described in claim 5, characterized in that, The preliminary transformed point cloud is corrected using the following formula: ； ； in, For the coarsely transformed point cloud, Let T be the initial transformed point cloud, and T be the transformation matrix.

7. The method as described in claim 1, characterized in that, Obtaining the point cloud depth maps of the source point cloud and the target point cloud includes: Point cloud clusters are aggregated for the source point cloud and the target point cloud respectively to obtain point cloud clusters of the source point cloud and the target point cloud, and the recognition box of each point cloud cluster is obtained. The identification box of the point cloud cluster is voxelized to obtain multiple voxels. The centroid of the midpoint of each voxel is taken as the voxel center point. The N voxel center points closest to the voxel center point form N curvature surfaces. The surface normal vectors of the N curvature surfaces are calculated. The curvature information of the neighborhood surface of the voxel center point is obtained based on the surface normal vectors. Based on the curvature information, the similarity between each point cloud cluster in the source point cloud and each point cloud cluster in the target point cloud is calculated. Based on the similarity, the most matching point cloud cluster and the most matching region in the target point cloud and the source point cloud are determined. The point cloud clusters in the point cloud with a large number of voxels in the most matching region in the source point cloud or the target point cloud are replaced by point cloud clusters in the point cloud with a small number of voxels. A point cloud depth map of the source point cloud is generated based on the point cloud clusters of the source point cloud. A point cloud depth map of the target point cloud is generated based on the point cloud clusters of the target point cloud.

8. The method as described in claim 7, characterized in that, N equals 3.

9. The method as described in claim 7 or 8, characterized in that, Obtaining the curvature information of the surface in the neighborhood of the voxel center point based on the surface normal vector includes: The average value of the surface normal vectors of the N curved surfaces is used as the curvature information of the neighborhood surface of the voxel center point.

10. The method as described in claim 7, characterized in that, Based on the curvature information, calculating the similarity between each point cloud cluster in the source point cloud and each point cloud cluster in the target point cloud includes: The voxel space with a smaller number of voxels in the voxel space corresponding to the source point cloud and the target point cloud is selected as the voxel convolution kernel. It is compared with the voxel space with a larger number of voxels to obtain the voxel coefficient of the same corresponding space. Based on the curvature information corresponding to the source point cloud and the target point cloud, calculate the curvature similarity coefficient between the source point cloud and the target point cloud; The similarity between the point cloud clusters in the source point cloud and the point cloud clusters in the target point cloud is calculated based on the corresponding space occupancy voxel coefficient and the curvature similarity coefficient.

11. The method as described in claim 10, characterized in that, The corresponding space-occupying voxel coefficients are calculated using the following formula: ； ； in, The corresponding space-occupying voxel coefficient is the same. The number of voxels in the voxel space of the point cloud with the smaller number of voxels. n is the number of voxels in the point cloud voxel space with the largest number of voxels, and n is the number of voxels with the same attribute in the voxel convolution kernel and the convolution region. The same attribute refers to point cloud voxels that are either points or no points.

12. The method as described in claim 10, characterized in that, The curvature similarity coefficient is calculated using the following formula: ； ； Where c is the curvature similarity coefficient. The curvature information corresponding to the source point cloud. The curvature information corresponding to the target point cloud. The number of voxels in the voxel space of the point cloud with the smaller number of voxels. The attributes of the voxels of the source point cloud, The attribute of the voxels of the target point cloud is whether the voxel contains points or not.

13. The method as described in claim 10, characterized in that, The similarity between point cloud clusters in the source point cloud and point cloud clusters in the target point cloud is calculated using the following formula: ； Where S is the similarity between the point cloud clusters in the source point cloud and the point cloud clusters in the target point cloud, and c is the curvature similarity coefficient. The corresponding space-occupying voxel coefficient is the same.

14. The method as described in claim 7, characterized in that, Generating a point cloud depth map of the source point cloud based on the point cloud clusters of the source point cloud includes: Calculate the centroid of each point cloud cluster in the source point cloud, and use the centroid of the point cloud cluster to represent the point cloud cluster; The directions of the eigenvectors corresponding to the two larger eigenvalues ​​among the three directions of the centroid are taken as the two axis directions of the coordinate system plane; Map the centroid of each point cloud cluster in the source point cloud to a planar grid in the coordinate system plane; Calculate the depth information of each grid point in the planar grid, and obtain the point cloud depth map of the source point cloud based on the depth information.

15. The method as described in claim 7, characterized in that, Generating a point cloud depth map of the target point cloud based on the point cloud clusters of the target point cloud includes: Calculate the centroid of each point cloud cluster in the target point cloud, and use the centroid of the point cloud cluster to represent the point cloud cluster; The directions of the eigenvectors corresponding to the two larger eigenvalues ​​among the three directions of the centroid are taken as the two axis directions of the coordinate system plane; Map the centroid of each point cloud cluster in the target point cloud to a planar grid in the coordinate system plane; Calculate the depth information of each grid point in the planar grid, and obtain the point cloud depth map of the target point cloud based on the depth information; Gaussian convolution is performed on the point cloud depth map to generate M Gaussian convolution point cloud depth maps.

16. The method as described in claim 15, characterized in that, The point cloud depth map is subjected to Gaussian convolution using the following formula: ； ； in, This is the depth map of the point cloud after Gaussian convolution. This is a point cloud depth map. = , This indicates the layer number of the point cloud depth map after Gaussian convolution, where (m,n) is the pixel size and (x,y) is the pixel coordinate.

17. The method as described in claim 1, characterized in that, The similarity of the feature point distribution descriptions of the source point cloud and the target point cloud can be expressed by the following formula: ； in, This indicates the similarity in feature point distribution descriptions between the source point cloud and the target point cloud. It is a constant. and These are the six-dimensional information of the regions in the feature point distribution feature descriptions of the source point cloud and the target point cloud, respectively. and These are the normalized normal vectors of the planes obtained by fitting the ground point clouds of the source point cloud and the target point cloud, respectively.

18. The method as described in claim 1, characterized in that, Determining the feature point matching relationship between the source point cloud and the target point cloud based on the feature points and their distribution characteristics includes: Depth map feature points are extracted from the depth maps of the source point cloud and the target point cloud to generate 256-dimensional binary depth map features of the feature points of the source point cloud and the target point cloud. The 256-dimensional binary depth map features of the feature points of the source point cloud and the target point cloud are matched to obtain the matching result. Based on the similarity of the feature point distribution features of the source point cloud and the target point cloud and the matching result, the feature point matching relationship between the source point cloud and the target point cloud is obtained.

19. An edge server, characterized in that, include: A coarse registration module is used to perform coarse registration between a source point cloud and a target point cloud to obtain a coarsely registered point cloud of the target point cloud. The source point cloud is a point cloud obtained by measurement using a first measuring device, and the target point cloud is a point cloud obtained by measurement using a second measuring device. The first measuring device and the second measuring device have a common measurement area. A precise registration module is used to precisely register the source point cloud and the coarse registration point cloud; The process of coarsely registering the source point cloud and the target point cloud to obtain the coarsely registered point cloud of the target point cloud includes: Obtain point cloud depth maps of the source point cloud and the target point cloud; Feature points are obtained from the depth maps of the source point cloud and the target point cloud, and the feature points are calculated to obtain a feature point distribution feature description of the source point cloud and the target point cloud. Based on the feature points and the feature point distribution feature description, determine the feature point matching relationship between the source point cloud and the target point cloud; Based on the feature point matching relationship, the transformation matrix of the target point cloud is obtained; Based on the transformation matrix, a coarsely matched point cloud is obtained; The calculation of the feature points to obtain the feature point distribution feature descriptions of the source point cloud and the target point cloud includes: Based on the feature points of the point cloud depth maps of the source point cloud and the target point cloud, find the spatial corresponding points. If there are multiple spatial corresponding points, retain the spatial corresponding points with the largest and smallest centroids at distances from the multiple spatial corresponding points. The R neighborhood points of the corresponding points in the space are counted, and the local plane coordinate system of the principal component analysis algorithm is constructed based on the R neighborhood points. The eigenvector with the smallest eigenvalue is used as the point curvature description, the first eigenvector and the second eigenvector of the principal component analysis algorithm are used as the direction, and the centroid of the point cloud cluster is used as the origin to form the local plane coordinate system. The local plane coordinate system is divided into two parts with the same width, and then further divided into 45-degree segments. The minimum 200-degree plane normal vector and the center point of each region are calculated to form the six-dimensional information of the region. The six-dimensional information of the region and the point curvature description are used as point feature information to generate a feature point distribution feature description.

20. An edge server, characterized in that, include: A processor, a memory, and a program stored in the memory and executable on the processor, wherein the program, when executed by the processor, implements the steps of the point cloud registration method as described in any one of claims 1 to 18.

21. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the point cloud registration method as described in any one of claims 1 to 18.

Images