Road surface traffic line point cloud registration method and device, electronic equipment and storage medium

Abstract

Embodiments of the present disclosure disclose a road traffic line point cloud registration method and device, electronic equipment and a storage medium, the method comprising: obtaining a source point cloud and a target point cloud, for each source point cloud point in the source point cloud, determining an associated point cloud point corresponding to the source point cloud point from the target point cloud according to the semantic label of the source point cloud point; for each traffic element in the point cloud set, determining an initial covariance matrix according to the coordinates of the point cloud points corresponding to the traffic element; singular value decomposition is performed on the initial covariance matrix to obtain a first matrix and a second matrix, and the target covariance matrix of each point cloud point in the traffic element is determined in combination with the element type of the traffic element; and a registration matrix is determined according to a preset target function, the target covariance matrix and the point cloud point coordinates of each source point cloud point, and the target covariance matrix and the point cloud point coordinates of the associated point cloud points respectively corresponding to each source point cloud point, so as to improve the point cloud registration speed and accuracy between road traffic marking line point clouds.

Description

Technical Field

[0001] This disclosure relates to the field of point cloud data processing, and in particular to a method, apparatus, electronic device, and storage medium for registering point clouds of road traffic lines. Background Technology

[0002] Road traffic lines contain high-level semantic information about the vehicle's current environment, aiding in environmental understanding, road planning, and localization mapping. In mapping tasks, fast and efficient point cloud registration methods can correctly stitch together multiple point cloud frames. In localization tasks, point cloud registration methods can solve for the spatial transformation relationships between point clouds acquired at different times, thereby estimating the vehicle's relative position and attitude over consecutive timeframes. Therefore, a good point cloud registration method contributes to improving mapping accuracy and vehicle localization precision.

[0003] Currently, point cloud registration methods are mainly divided into three categories: methods based on normal distribution transformation, data-driven methods, and nearest-point iteration methods. Methods based on normal distribution transformation are highly sensitive to grid size and point cloud density, making them unsuitable for registering variable road line point clouds. Data-driven methods require extensive pre-training, but road traffic sign point clouds are specific to ordinary scene point clouds, and models trained using ordinary point clouds are often unsuitable for registering road traffic line point clouds. Nearest-point iteration methods ignore the constraints imposed by lane endpoints on the transformation solution, leading to degenerate results. Furthermore, the lack of planar constraints in road traffic line point clouds further contributes to the poor performance of this method. Summary of the Invention

[0004] To address the aforementioned technical problems, or at least partially address them, this disclosure provides a method, apparatus, electronic device, and storage medium for registering road traffic line point clouds, thereby improving the speed and accuracy of point cloud registration between road traffic marking point clouds.

[0005] In a first aspect, embodiments of this disclosure provide a method for registering point clouds of road traffic lines, the method comprising:

[0006] Obtain the source point cloud corresponding to the source road traffic line and the target point cloud corresponding to the target road traffic line. For each source point cloud point in the source point cloud, determine the associated point cloud point corresponding to the source point cloud point from the target point cloud based on the semantic label of the source point cloud point.

[0007] For each traffic element in the point cloud set, the initial covariance matrix of the traffic element is determined based on the coordinates of each point cloud point corresponding to the traffic element; wherein, the point cloud set includes the source point cloud and the target point cloud;

[0008] Singular value decomposition is performed on the initial covariance matrix to obtain a first matrix and a second matrix. Based on the element type of the traffic element, the first matrix, and the second matrix, the target covariance matrix corresponding to each point cloud point in the traffic element is determined.

[0009] Based on the preset objective function, the objective covariance matrix of each source point cloud point, the point cloud coordinates of each source point cloud point, the objective covariance matrix of the associated point cloud points corresponding to each source point cloud point, and the point cloud coordinates of the associated point cloud points, the registration matrix between the source point cloud and the target point cloud is determined; wherein, the preset objective function is the objective function in the generalized nearest point iteration algorithm.

[0010] Secondly, this disclosure also provides a road traffic line point cloud registration device, the device comprising:

[0011] The association module is used to obtain the source point cloud corresponding to the source road traffic line and the target point cloud corresponding to the target road traffic line. For each source point cloud point in the source point cloud, the associated point cloud point corresponding to the source point cloud point is determined from the target point cloud based on the semantic label of the source point cloud point.

[0012] The initial covariance matrix determination module is used to determine the initial covariance matrix of each traffic element in the point cloud set based on the coordinates of each point cloud point corresponding to the traffic element; wherein, the point cloud set includes the source point cloud and the target point cloud.

[0013] The target covariance matrix determination module is used to perform singular value decomposition on the initial covariance matrix to obtain a first matrix and a second matrix, and determine the target covariance matrix corresponding to each point cloud point in the traffic element according to the element type of the traffic element, the first matrix and the second matrix;

[0014] The registration matrix determination module is used to determine the registration matrix between the source point cloud and the target point cloud based on a preset objective function, the target covariance matrix of each source point cloud point, the point cloud coordinates of each source point cloud point, the target covariance matrix of the associated point cloud points corresponding to each source point cloud point, and the point cloud coordinates of the associated point cloud points; wherein, the preset objective function is the objective function in the generalized nearest point iteration algorithm.

[0015] Thirdly, this disclosure also provides an electronic device, which includes: one or more processors; a storage device for storing one or more programs; and when the one or more programs are executed by the one or more processors, the one or more processors implement the road traffic line point cloud registration method as described above.

[0016] Fourthly, embodiments of this disclosure also provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the road traffic line point cloud registration method as described above.

[0017] This disclosure provides a method for registering road traffic line point clouds. It involves acquiring a source point cloud corresponding to a source road traffic line and a target point cloud corresponding to a target road traffic line. For each source point cloud point, based on its semantic label, it determines the associated point cloud point from the target point cloud. This semantic label association between the source and target point cloud points is then used. Furthermore, for each traffic element in the point cloud set including both the source and target point clouds, it determines the initial covariance matrix of the traffic element based on the coordinates of the corresponding point cloud points. Singular value analysis is then performed on the initial covariance matrix. The solution yields the first and second matrices. Based on the element type of the traffic element, the first and second matrices, the target covariance matrix corresponding to each point cloud point in the traffic element is determined to simplify the solution of the target covariance matrix. Furthermore, based on the preset objective function, the target covariance matrix of each source point cloud point, the point cloud coordinates of each source point cloud point, the target covariance matrix of the associated point cloud points corresponding to each source point cloud point, and the point cloud coordinates of the associated point cloud points, the registration matrix between the source point cloud and the target point cloud is determined. This achieves the effect of improving the speed and accuracy of point cloud registration between road traffic marking point clouds. Attached Figure Description

[0018] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale.

[0019] Figure 1 This is a flowchart of a point cloud registration method for road traffic lines in an embodiment of this disclosure;

[0020] Figure 2 This is a schematic diagram of the structure of a road traffic line point cloud registration device in an embodiment of this disclosure;

[0021] Figure 3 This is a schematic diagram of the structure of an electronic device according to an embodiment of this disclosure. Detailed Implementation

[0022] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0023] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0024] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0025] Figure 1 This is a flowchart illustrating a method for registering road traffic line point clouds according to an embodiment of this disclosure. This method can be executed by a road traffic line point cloud registration device, which can be implemented in software and / or hardware and can be configured in an electronic device. Figure 1 As shown, the method may specifically include the following steps:

[0026] S110. Obtain the source point cloud corresponding to the source road traffic line and the target point cloud corresponding to the target road traffic line. For each source point cloud point in the source point cloud, determine the associated point cloud point corresponding to the source point cloud point from the target point cloud based on the semantic label of the source point cloud point.

[0027] The source point cloud and the target point cloud are two sets of point clouds that need to be registered. The source and target point clouds have overlapping parts; by applying the obtained registration matrix to the source point cloud, the overlapping parts can be registered as closely as possible. The source and target point clouds can be acquired using a point cloud acquisition device. The source road traffic lines are the road traffic lines corresponding to the source point cloud, and the target road traffic lines are the road traffic lines corresponding to the target point cloud. Source point cloud points are the points in the source point cloud, and target point cloud points are the points in the target point cloud. The semantic label is the road traffic line type of the traffic element to which the point cloud point belongs. A traffic element can be understood as a road traffic line. The associated point cloud point is the target point cloud point that is closest to a point in the source point cloud.

[0028] Specifically, two sets of point clouds need to be acquired for point cloud registration: one set is the source point cloud corresponding to the source road traffic lines, and the other set is the target point cloud corresponding to the target road traffic lines. Then, for each source point cloud point, the associated target point cloud point can be determined from the target point cloud in the same way. Taking one source point cloud point as an example, for each source point cloud point, based on its semantic label, all target point cloud points with the same semantic label are found in the target point cloud. The nearest neighbor among these target point cloud points is then selected as the associated point cloud point corresponding to the source point cloud point.

[0029] Based on the above example, the associated point cloud points corresponding to the source point cloud points can be determined from the target point cloud using the semantic labels of the source point cloud points in the following way:

[0030] Based on the target point cloud points corresponding to each semantic label in the target point cloud, construct a k-dimensional tree corresponding to each semantic label;

[0031] For each source point in the source point cloud, based on the semantic label corresponding to the source point, the target tree is determined from each k-dimensional tree, and the target point in the target tree that is closest to the source point is taken as the associated point in the source point.

[0032] Here, a k-dimensional tree is a tree-like data structure used to store points in a k-dimensional space for fast retrieval. The target number is a k-dimensional tree with the same semantic labels as the source point cloud points.

[0033] Specifically, for each semantic label in the target point cloud, all target point cloud points are extracted. For each semantic label, a k-dimensional tree (KD-Tree) is constructed based on all target point cloud points corresponding to that semantic label. For each source point cloud point, a corresponding target point cloud point needs to be searched from all target point cloud points with the same semantic label. Therefore, according to the semantic label corresponding to the source point cloud point, a k-dimensional tree with the same semantic label is determined from all k-dimensional trees as the target tree. The target point cloud point closest to the source point cloud point is searched from the target tree as the associated point cloud point corresponding to the source point cloud point. If no k-dimensional tree with the same semantic label as the source point cloud point exists, the source point cloud point does not participate in subsequent steps.

[0034] S120. For each traffic element in the point cloud set, determine the initial covariance matrix of the traffic element based on the coordinates of each point cloud point corresponding to the traffic element.

[0035] The point cloud set includes both source and target point clouds. Point cloud coordinates are the coordinates of point clouds within the point cloud coordinate system. The initial covariance matrix is ​​a matrix composed of the covariances between the point clouds in the traffic element.

[0036] Specifically, for each traffic element in the point cloud set, that is, for each traffic element in the source point cloud and the target point cloud, it is necessary to calculate the covariance between the coordinates of any two point cloud points in the traffic element, so as to obtain the covariance matrix, which can be used as the initial covariance matrix of the traffic element.

[0037] Based on the above example, the initial covariance matrix of a traffic element can be determined using the coordinates of each point cloud point corresponding to that traffic element, as follows:

[0038] The point cloud center is determined based on the coordinates of each point cloud point corresponding to the traffic element and the number of point cloud points corresponding to the traffic element.

[0039] The initial covariance matrix of the traffic element is determined based on the point cloud center, the coordinates of each point cloud point corresponding to the traffic element, and the number of point cloud points corresponding to the traffic element.

[0040] Here, the number of point cloud points is the total number of point cloud points in a traffic element. The point cloud center is the center of each point cloud point corresponding to a traffic element.

[0041] Specifically, for each traffic element, the center of the point cloud is obtained by analyzing the positions of the corresponding point cloud points, i.e., by combining the coordinates of the corresponding point cloud points and the number of corresponding point cloud points. Further, the covariance matrix is ​​calculated using the obtained point cloud center, the coordinates of the corresponding point cloud points, and the number of corresponding point cloud points, and this covariance matrix serves as the initial covariance matrix for the traffic element.

[0042] Based on the above example, the point cloud center can be determined using the coordinates of each point cloud point corresponding to a traffic element and the number of point cloud points corresponding to the traffic element, as follows:

[0043] The center of the point cloud is determined using the following formula:

[0044]

[0045] Where, μ k Let n be the center of the point cloud for the k-th traffic element. k Let p(i,j) be the number of point cloud points corresponding to the k-th traffic element, and p(i,j) be the coordinates of each point cloud point corresponding to the k-th traffic element.

[0046] Based on the above example, the initial covariance matrix of a traffic element can be determined using the following method: the point cloud center, the coordinates of each point cloud point corresponding to the traffic element, and the number of point cloud points corresponding to the traffic element.

[0047] The initial covariance matrix of the traffic elements is determined using the following formula:

[0048]

[0049] Among them, C k Let μ be the initial covariance matrix of the k-th traffic element. k Let n be the center of the point cloud for the k-th traffic element. k Let p(i,j) be the number of point cloud points corresponding to the k-th traffic element, and p(i,j) be the coordinates of each point cloud point corresponding to the k-th traffic element.

[0050] S130. Perform singular value decomposition on the initial covariance matrix to obtain the first matrix and the second matrix. Based on the element type of the traffic element, the first matrix, and the second matrix, determine the target covariance matrix corresponding to each point cloud point in the traffic element.

[0051] The element type describes the shape of traffic elements, such as long line type, line segment type, and other types. The first and second matrices are obtained by performing singular value decomposition on the initial covariance matrix. The target covariance matrix is ​​the covariance matrix obtained by standardizing the initial covariance matrix based on the differential manifold.

[0052] Since the calculated initial covariance matrix may contain extremely large differences in singular values, this can lead to problems such as numerical precision overflow during the subsequent solution of the objective function. Therefore, it is necessary to construct a differential manifold to standardize the initial covariance matrix and obtain the target covariance matrix.

[0053] Specifically, singular value decomposition is performed on the initial covariance matrix to obtain the first and second matrices:

[0054]

[0055] Where C is the initial covariance matrix of the traffic elements, U is the first matrix, and V is the second matrix. It is a matrix in which all elements on the main diagonal are singular values ​​of the initial covariance matrix C, and all elements outside the main diagonal are 0. For example, Where λ3≤λ2≤λ1, and λ1, λ2 and λ3 are all singular values ​​of the initial covariance matrix C.

[0056] Furthermore, based on the element type of the traffic element, a diagonal matrix composed of eigenvalues ​​is determined. Multiplying the first matrix, the diagonal matrix composed of the eigenvalues, and the second matrix yields the standardized covariance matrix, which is the target covariance matrix corresponding to each point cloud point in the traffic element.

[0057] Based on the above example, the target covariance matrix corresponding to each point cloud point in the traffic element can be determined according to the element type of the traffic element, the first matrix, and the second matrix in the following way:

[0058] Determine the type parameter based on the element type of the traffic element;

[0059] Construct a diagonal matrix based on preset parameters and type parameters;

[0060] The product of the first matrix, the diagonal matrix, and the second matrix is ​​used as the target covariance matrix for each point cloud point in the traffic element.

[0061] The type parameter is a pre-defined parameter corresponding to different element types. The preset parameter is a pre-defined parameter used to represent minute constants. The diagonal matrix is ​​a matrix with the preset parameter and the type parameter as elements on the main diagonal.

[0062] Specifically, based on the element type of the traffic element, the type parameter corresponding to that traffic element can be determined from the pre-defined parameters corresponding to each element type. By taking the type parameter and the pre-defined parameter as the last two elements on the main diagonal, and setting the remaining elements on the main diagonal to 1, a diagonal matrix can be obtained. The product of the first matrix, the diagonal matrix, and the second matrix is ​​used as the standardized covariance matrix, which is the target covariance matrix corresponding to each point cloud point in that traffic element.

[0063] Based on the example above, the product of the first matrix, the diagonal matrix, and the second matrix can be used as the target covariance matrix for each point cloud point in the traffic element in the following way:

[0064] The target covariance matrix corresponding to each point in the traffic element is determined by the following formula:

[0065] C′=U∑′V

[0066] Where C′ is the target covariance matrix corresponding to each point cloud point in the traffic element, U is the first matrix, V is the second matrix, ∑′ is the diagonal matrix, ∑′=diag(1,δ,∈), δ is the type parameter, and ∈ is the preset parameter.

[0067] It can be seen that the target covariance matrix determined by the above method includes: the target covariance matrix C corresponding to each source point cloud point in the source point cloud.s ′ and the target covariance matrix C corresponding to each target point in the target point cloud. t ′.

[0068] Based on the above example, the type parameter can be determined according to the element type of the traffic element in the following way:

[0069] If the element type of the traffic element is long line, then the type parameter is determined to be the preset parameter;

[0070] If the element type of the traffic element is other types, then the type parameter is set to 1;

[0071] If the element type of a traffic element is line segment, then the type parameter is determined to be a constraint parameter.

[0072] Among them, the constraint parameter is greater than the preset parameter and the constraint parameter is less than 1.

[0073] Traffic elements, namely road markings, can be divided into three types: long lines, line segments, and other types. Long lines refer to linear elements whose endpoints cannot be consistently observed within the local observation range of a vehicle, such as solid lane lines and curbs. The point cloud of these elements exhibits a linear distribution and lacks clear endpoints within a local area. Line segments refer to linearly distributed elements whose endpoints can be observed, such as dashed lane lines, stop lines, and pedestrian crossing markings. All other traffic elements besides the above two types are classified as other elements. Their point cloud distribution does not exhibit a clear pattern and often appears in geometric shapes, such as speed limit signs, triangular signs (deceleration signs), and diamond-shaped signs (pedestrian crossing warning signs).

[0074] Specifically, for long lines, due to their continuity, their endpoints may not be effectively observed in a single frame of the point cloud. Therefore, constructing nearest-neighbor constraints along the line direction for these point cloud points cannot reduce the optimization objective; that is, the constraints along the line direction for long lines are considered weak. In this case, a differential flow-type covariance matrix needs to be constructed, where the type parameter δ is equal to the preset parameter ∈, where ∈ is a small constant. To ensure that precision overflow does not occur in the numerical calculations on the computer, ∈ can be set to 10. -3In this case, for point cloud points along a long line, the distance between the nearest neighbor pairs along the line direction does not significantly affect the descent of the optimization objective. However, matching point cloud pairs perpendicular to the long line direction (source point cloud point and its corresponding associated point cloud point) will make the optimization objective descent faster, achieving the goal of weakening the constraints along the line direction while strengthening the constraints perpendicular to the long line direction. For other elements such as diamond-shaped markings and speed limit markings, their point cloud points are usually distributed planarly on the road surface, and the distribution on the plane is relatively dispersed rather than concentrated in a linear distribution. Therefore, for this type, a two-differential flow covariance matrix is ​​constructed, i.e., the type parameter δ = 1. In this case, the distance between point cloud pairs (source point cloud point and its corresponding associated point cloud point) in other types of traffic elements has the same constraint ability on the plane, strengthening the constraint on the direction perpendicular to the plane (road surface). For line segment types, the endpoints of the line segment can usually be observed in a single frame of point cloud. Therefore, compared to long lines, the endpoints of such elements can provide certain constraints along the line direction. However, compared to traffic markings with geometric shapes, line segment types have weaker constraints on the direction along the line. Therefore, the type parameter δ needs to be constructed to satisfy ∈ <δ < 1, ensuring that when only long line and line segment type traffic elements exist in the same direction, the point cloud portion of the line segment type provides stronger constraints than the long line type. Furthermore, when other types of geometric traffic elements exist, the matching of point cloud pairs in the line segment type traffic elements should not excessively affect the overall registration of the point cloud. For example, δ = 10 can be chosen. -1 As the type parameter corresponding to the line segment type.

[0075] S130 can be used to obtain the target covariance matrix of each source point cloud point in the source point cloud and the target covariance matrix of each target point cloud point in the target point cloud.

[0076] S140. Based on the preset objective function, the objective covariance matrix of each source point cloud point, the point cloud coordinates of each source point cloud point, the objective covariance matrix of the associated point cloud points corresponding to each source point cloud point, and the point cloud coordinates of the associated point cloud points, determine the registration matrix between the source point cloud and the target point cloud.

[0077] The preset objective function is the objective function in the generalized nearest-point iterative algorithm. The registration matrix is ​​a matrix applied to the source point cloud to register the overlapping portion of the source and target point clouds.

[0078] The preset objective function is:

[0079]

[0080] in, and Let C be the coordinates of the i-th source point cloud point and the coordinates of the associated point cloud point, respectively, which are the coordinates of the two nearest neighbor point cloud point pairs with the same label in both the source and target point clouds. Let n be the number of point cloud points in the source point cloud. s ′ is the target covariance matrix of the i-th source point cloud, C t ′ is the target covariance matrix of the associated point cloud points corresponding to the i-th source point cloud point. T * This is the registration matrix, i.e., the optimization result corresponding to the preset objective function. T is the objective to be optimized, i.e., the registration matrix to be optimized, which is a 4×4 rigid body transformation matrix that satisfies... Where R is the rotation matrix with a size of 3×3, and t is the translation vector with a size of 1×3.

[0081] By solving for the preset objective function, a registration matrix is ​​found that minimizes the value of the preset objective function, i.e., minimizes the sum of distances between point cloud pairs (source point cloud points and their corresponding associated point cloud points). For this purpose, commonly used nonlinear optimization algorithms can be selected, such as gradient descent, Gauss-Newton method, or Levenberg-Marquardt method (LM method). This yields the result of the current iteration, i.e., the registration matrix. Next, the obtained registration matrix is ​​used to perform rotation and translation transformations on the source point cloud to obtain a new source point cloud. The registration effect between the new source point cloud and the target point cloud is then determined. If it meets the preset requirements, the registration is complete; otherwise, the next iteration begins based on the new source point cloud. This process continues until the maximum number of iterations is reached or the registration effect between the new source point cloud and the target point cloud meets the preset requirements. Finally, the registration result between the two frames of point clouds (source and target point clouds) is obtained.

[0082] The generalized nearest-point iterative algorithm requires pre-estimating the normal vector of each point cloud point. This typically involves fitting plane parameters using multiple points within a local region, and then using the normal vector of that plane as an estimate of the point cloud point's normal vector. As the number of point cloud points increases, the processing time increases significantly, resulting in low real-time performance in dense point cloud registration. The road traffic line point cloud registration method provided in this embodiment uses the covariance matrix of each traffic element to approximately fit the target covariance matrix of each point cloud point within that traffic element. This not only reduces computational load and improves the efficiency of point cloud registration, but also approximates the spatial distribution pattern of each point cloud point as closely as possible.

[0083] Compared to other nearest-point iteration-based methods, the road traffic line point cloud registration method provided in this embodiment is specifically optimized for the distribution characteristics (element types of traffic elements) of the road traffic marking point cloud, especially for cases with only lane line point clouds. It can utilize different differential flow model construction methods to reduce the convergence amplitude along weakly constrained directions, avoiding solution degradation. Compared to methods based on normal distribution transformation, the road traffic line point cloud registration method provided in this embodiment does not require constructing the target point cloud into a raster form, thus achieving higher matching efficiency. Furthermore, the road traffic line point cloud registration method provided in this embodiment does not require manually setting prior parameters such as raster size for point cloud density and type, reducing reliance on manual intervention. Compared to data-driven methods, pre-trained point cloud registration models are not suitable for registering road traffic marking point clouds with specific characteristics. The road traffic line point cloud registration method provided in this embodiment does not require manual annotation and training with large amounts of data beforehand, reducing labor and time costs.

[0084] The road traffic line point cloud registration method provided in this embodiment obtains the source point cloud corresponding to the source road traffic line and the target point cloud corresponding to the target road traffic line. For each source point cloud point in the source point cloud, based on the semantic label of the source point cloud point, it determines the associated point cloud point corresponding to the source point cloud point from the target point cloud, thus associating the source point cloud point and the target point cloud point by semantic label. Furthermore, for each traffic element in the point cloud set including the source point cloud and the target point cloud, it determines the initial covariance matrix of the traffic element based on the coordinates of the corresponding point cloud points, and performs singular value decomposition on the initial covariance matrix. The first and second matrices are obtained. Based on the element type of the traffic element, the first and second matrices, the target covariance matrix corresponding to each point cloud point in the traffic element is determined to simplify the solution of the target covariance matrix. Then, based on the preset objective function, the target covariance matrix of each source point cloud point, the point cloud coordinates of each source point cloud point, the target covariance matrix of the associated point cloud points corresponding to each source point cloud point, and the point cloud coordinates of the associated point cloud points, the registration matrix between the source point cloud and the target point cloud is determined, thereby improving the speed and accuracy of point cloud registration between road traffic marking point clouds.

[0085] If the above-mentioned road traffic line point cloud registration method is used in the mapping task of autonomous driving, it can complete the stitching of multi-frame point clouds with higher accuracy through point cloud registration, thereby generating a higher accuracy semantic point cloud map. If the above-mentioned road traffic line point cloud registration method is used in the localization task of autonomous driving, it can solve the vehicle pose by registering multiple consecutive frames of road traffic marking point clouds, thereby achieving high-precision tracking of vehicle localization.

[0086] Figure 2 This is a schematic diagram of the structure of a road traffic line point cloud registration device according to an embodiment of this disclosure. Figure 2 As shown, the device includes: an association module 210, an initial covariance matrix determination module 220, a target covariance matrix determination module 230, and a registration matrix determination module 240.

[0087] The association module 210 is used to acquire the source point cloud corresponding to the source road traffic line and the target point cloud corresponding to the target road traffic line. For each source point cloud point in the source point cloud, it determines the associated point cloud point corresponding to the source point cloud point from the target point cloud based on the semantic label of the source point cloud point. The initial covariance matrix determination module 220 is used to determine the initial covariance matrix of each traffic element in the point cloud set based on the coordinates of each point cloud point corresponding to the traffic element. The point cloud set includes the source point cloud and the target point cloud. The target covariance matrix determination module 230 is used to determine the initial covariance matrix of the target point cloud. The difference matrix is ​​subjected to singular value decomposition to obtain a first matrix and a second matrix. Based on the element type of the traffic element, the first matrix, and the second matrix, the target covariance matrix corresponding to each point cloud point in the traffic element is determined. The registration matrix determination module 240 is used to determine the registration matrix between the source point cloud and the target point cloud based on a preset objective function, the target covariance matrix of each source point cloud point, the point cloud coordinates of each source point cloud point, the target covariance matrix of the associated point cloud points corresponding to each source point cloud point, and the point cloud coordinates of the associated point cloud points. The preset objective function is the objective function in the generalized nearest point iteration algorithm.

[0088] Based on the above example, optionally, the initial covariance matrix determination module 220 is further configured to determine the point cloud center based on the coordinates of each point cloud point corresponding to the traffic element and the number of point cloud points corresponding to the traffic element; and to determine the initial covariance matrix of the traffic element based on the point cloud center, the coordinates of each point cloud point corresponding to the traffic element and the number of point cloud points corresponding to the traffic element.

[0089] Based on the above example, optionally, the initial covariance matrix determination module 220 is also used to determine the center of the point cloud using the following formula:

[0090]

[0091] Where, μ k Let n be the center of the point cloud for the k-th traffic element. k p(i,j) represents the number of point cloud points corresponding to the k-th traffic element, and p(i,j) represents the coordinates of each point cloud point corresponding to the k-th traffic element.

[0092] Accordingly, the initial covariance matrix determination module 220 is also used to determine the initial covariance matrix of the traffic element using the following formula:

[0093]

[0094] Among them, C k Let μ be the initial covariance matrix of the k-th traffic element. k Let n be the center of the point cloud for the k-th traffic element. k Let p(i,j) be the number of point cloud points corresponding to the k-th traffic element, and p(i,j) be the coordinates of each point cloud point corresponding to the k-th traffic element.

[0095] Based on the above example, optionally, the target covariance matrix determination module 230 is further configured to determine a type parameter according to the element type of the traffic element; construct a diagonal matrix according to a preset parameter and the type parameter; and use the product of the first matrix, the diagonal matrix and the second matrix as the target covariance matrix corresponding to each point cloud point in the traffic element.

[0096] Based on the above example, optionally, the target covariance matrix determination module 230 is further configured to determine the type parameter as the preset parameter if the element type of the traffic element is a long line type; determine the type parameter as 1 if the element type of the traffic element is another type; and determine the type parameter as a constraint parameter if the element type of the traffic element is a line segment type; wherein the constraint parameter is greater than the preset parameter and the constraint parameter is less than 1.

[0097] Based on the above example, optionally, the target covariance matrix determination module 230 is further configured to determine the target covariance matrix corresponding to each point cloud point in the traffic element using the following formula:

[0098] C ′ =UΣ ′ V

[0099] Among them, C ′ Let U be the target covariance matrix corresponding to each point cloud point in the traffic element, V be the first matrix, and Σ be the second matrix. ′ Let Σ be a diagonal matrix. ′ =diag(1,δ,∈), where δ is the type parameter and ∈ is the preset parameter.

[0100] Based on the above example, optionally, the association module 210 is further configured to construct a k-dimensional tree corresponding to each semantic label based on the target point cloud points in the target point cloud corresponding to each semantic label; for each source point cloud point in the source point cloud, a target tree is determined from each of the k-dimensional trees based on the semantic label corresponding to the source point cloud point, and the target point cloud point in the target tree that is closest to the source point cloud point is taken as the associated point cloud point corresponding to the source point cloud point.

[0101] The road traffic line point cloud registration device provided in this embodiment can execute the steps in the road traffic line point cloud registration method provided in this embodiment, and has the execution steps and beneficial effects, which will not be repeated here.

[0102] Figure 3 This is a schematic diagram of the structure of an electronic device according to an embodiment of this disclosure. See below for details. Figure 3 It shows a schematic diagram of a structure suitable for implementing the electronic device 300 in the embodiments of this disclosure. Figure 3 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0103] like Figure 3 As shown, the electronic device 300 may include a processing device 301, a read-only memory (ROM) 302, a random access memory (RAM) 303, a bus 304, an input / output (I / O) interface 305, an input device 306, an output device 307, a storage device 308, and a communication device 309. The processing device (e.g., a central processing unit, a graphics processor, etc.) 301 can perform various appropriate actions and processes to implement the methods of the embodiments described in this disclosure, based on a program in the ROM 302 or a program loaded from the storage device 308 into the RAM 303. The RAM 303 also stores various programs and data required for the operation of the electronic device 300. The processing device 301, the ROM 302, and the RAM 303 are interconnected via the bus 304. The input / output (I / O) interface 305 is also connected to the bus 304.

[0104] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts, thereby implementing the road traffic line point cloud registration method as described above. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 309, or installed from a storage device 308, or installed from a ROM 302. When the computer program is executed by the processing device 301, it performs the functions defined in the methods of embodiments of this disclosure.

[0105] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0106] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device. The aforementioned computer-readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to:

[0107] Obtain the source point cloud corresponding to the source road traffic line and the target point cloud corresponding to the target road traffic line. For each source point cloud point in the source point cloud, determine the associated point cloud point corresponding to the source point cloud point from the target point cloud based on the semantic label of the source point cloud point.

[0108] For each traffic element in the point cloud set, the initial covariance matrix of the traffic element is determined based on the coordinates of each point cloud point corresponding to the traffic element; wherein, the point cloud set includes the source point cloud and the target point cloud;

[0109] Singular value decomposition is performed on the initial covariance matrix to obtain a first matrix and a second matrix. Based on the element type of the traffic element, the first matrix, and the second matrix, the target covariance matrix corresponding to each point cloud point in the traffic element is determined.

[0110] Based on the preset objective function, the objective covariance matrix of each source point cloud point, the point cloud coordinates of each source point cloud point, the objective covariance matrix of the associated point cloud points corresponding to each source point cloud point, and the point cloud coordinates of the associated point cloud points, the registration matrix between the source point cloud and the target point cloud is determined; wherein, the preset objective function is the objective function in the generalized nearest point iteration algorithm.

[0111] Optionally, when one or more of the above-described procedures are executed by the electronic device, the electronic device may also execute other steps described in the above embodiments.

[0112] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0113] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

Claims

1. A method for registering point clouds of road traffic lines, characterized in that, The method includes: Obtain the source point cloud corresponding to the source road traffic line and the target point cloud corresponding to the target road traffic line. For each source point cloud point in the source point cloud, determine the associated point cloud point corresponding to the source point cloud point from the target point cloud based on the semantic label of the source point cloud point. For each traffic element in the point cloud set, the initial covariance matrix of the traffic element is determined based on the coordinates of each point cloud point corresponding to the traffic element; wherein, the point cloud set includes the source point cloud and the target point cloud; Singular value decomposition is performed on the initial covariance matrix to obtain a first matrix and a second matrix. Based on the element type of the traffic element, the first matrix, and the second matrix, the target covariance matrix corresponding to each point cloud point in the traffic element is determined. Based on the preset objective function, the objective covariance matrix of each source point cloud point, the point cloud coordinates of each source point cloud point, the objective covariance matrix of the associated point cloud points corresponding to each source point cloud point, and the point cloud coordinates of the associated point cloud points, the registration matrix between the source point cloud and the target point cloud is determined; wherein, the preset objective function is the objective function in the generalized nearest point iteration algorithm.

2. The method according to claim 1, characterized in that, The step of determining the initial covariance matrix of the traffic element based on the coordinates of each point cloud point corresponding to the traffic element includes: The point cloud center is determined based on the coordinates of each point cloud point corresponding to the traffic element and the number of point cloud points corresponding to the traffic element. The initial covariance matrix of the traffic element is determined based on the point cloud center, the coordinates of each point cloud point corresponding to the traffic element, and the number of point cloud points corresponding to the traffic element.

3. The method according to claim 2, characterized in that, The step of determining the point cloud center based on the coordinates of each point cloud point corresponding to the traffic element and the number of point cloud points corresponding to the traffic element includes: The center of the point cloud is determined using the following formula: Where, μ k Let n be the center of the point cloud for the k-th traffic element. k p(i,j) represents the number of point cloud points corresponding to the k-th traffic element, and p(i,j) represents the coordinates of each point cloud point corresponding to the k-th traffic element. Accordingly, determining the initial covariance matrix of the traffic element based on the point cloud center, the coordinates of each point cloud point corresponding to the traffic element, and the number of point cloud points corresponding to the traffic element includes: The initial covariance matrix of the traffic elements is determined by the following formula: Among them, C k Let μ be the initial covariance matrix of the k-th traffic element. k Let n be the center of the point cloud for the k-th traffic element. k Let p(i,j) be the number of point cloud points corresponding to the k-th traffic element, and p(i,j) be the coordinates of each point cloud point corresponding to the k-th traffic element.

4. The method according to claim 1, characterized in that, The step of determining the target covariance matrix corresponding to each point cloud point in the traffic element based on the element type of the traffic element, the first matrix, and the second matrix includes: Determine the type parameter based on the element type of the traffic element; Construct a diagonal matrix based on the preset parameters and the type parameters; The product of the first matrix, the diagonal matrix, and the second matrix is ​​used as the target covariance matrix corresponding to each point cloud point in the traffic element.

5. The method according to claim 4, characterized in that, The step of determining the type parameter based on the element type of the traffic element includes: If the element type of the traffic element is a long line, then the type parameter is determined to be the preset parameter; If the element type of the traffic element is other types, then the type parameter is determined to be 1; If the element type of the traffic element is a line segment, then the type parameter is determined to be a constraint parameter; wherein the constraint parameter is greater than the preset parameter and the constraint parameter is less than 1.

6. The method according to claim 5, characterized in that, The step of using the product of the first matrix, the diagonal matrix, and the second matrix as the target covariance matrix corresponding to each point cloud point in the traffic element includes: The target covariance matrix corresponding to each point cloud point in the traffic element is determined by the following formula: C′=U∑′V Where C′ is the target covariance matrix corresponding to each point cloud point in the traffic element, U is the first matrix, V is the second matrix, ∑′ is the diagonal matrix, ∑′=diag(1,δ,∈), δ is the type parameter, and ∈ is the preset parameter.

7. The method according to claim 1, characterized in that, The step of determining the associated point cloud points corresponding to the source point cloud points from the target point cloud based on the semantic tags of the source point cloud points includes: Based on the target point cloud points corresponding to each semantic label in the target point cloud, construct a k-dimensional tree corresponding to each semantic label; For each source point in the source point cloud, a target tree is determined from each k-dimensional tree based on the semantic label corresponding to the source point. The target point in the target tree that is closest to the source point is taken as the associated point in the source point.

8. A point cloud registration device for road traffic lines, characterized in that, include: The association module is used to obtain the source point cloud corresponding to the source road traffic line and the target point cloud corresponding to the target road traffic line. For each source point cloud point in the source point cloud, the associated point cloud point corresponding to the source point cloud point is determined from the target point cloud based on the semantic label of the source point cloud point. The initial covariance matrix determination module is used to determine the initial covariance matrix of each traffic element in the point cloud set based on the coordinates of each point cloud point corresponding to the traffic element; wherein, the point cloud set includes the source point cloud and the target point cloud. The target covariance matrix determination module is used to perform singular value decomposition on the initial covariance matrix to obtain a first matrix and a second matrix, and determine the target covariance matrix corresponding to each point cloud point in the traffic element according to the element type of the traffic element, the first matrix and the second matrix; The registration matrix determination module is used to determine the registration matrix between the source point cloud and the target point cloud based on a preset objective function, the target covariance matrix of each source point cloud point, the point cloud coordinates of each source point cloud point, the target covariance matrix of the associated point cloud points corresponding to each source point cloud point, and the point cloud coordinates of the associated point cloud points; wherein, the preset objective function is the objective function in the generalized nearest point iteration algorithm.

9. An electronic device, characterized in that, The electronic device includes: One or more processors; Storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the road traffic line point cloud registration method as described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the road traffic line point cloud registration method as described in any one of claims 1-7.

Images