A Dynamic Projection Method and System for On-vehicle Laser Point Cloud Data

Abstract

The present invention discloses a dynamic projection method and system for on-vehicle lidar point cloud data. Point cloud data is obtained through on-vehicle lidar. First, following the principle that points closer to the lidar have a higher density than those in the distance, the radar scan trajectory is identified by detecting areas with high point cloud density. Secondly, the projection direction is aligned with the radar scan direction, and dynamic coordinate axes and horizontal cylindrical projections are adaptively constructed for each point cloud area to obtain fitting projection parameters (such as radius, center point, height, etc.). The present invention better enhances the retention of ground object features in dense point clouds, greatly alleviates the deformation of ground objects during the projection process, has less occlusion between ground objects, reduces semantic confusion, improves the effect of the point cloud semantic segmentation network, and enhances the artificial intelligence inference speed, which can contribute to the development of autonomous driving map construction, three-dimensional model reconstruction, and smart cities, etc.

Description

Technical Field

[0001] The present invention belongs to the technical field of point cloud data projection, and particularly relates to a dynamic projection method and system for vehicle-mounted laser point cloud data. Background Technique

[0002] Currently, laser point cloud data is the main data source for the production of road three-dimensional spatio-temporal data. The idea of intelligent data production technology is mainly to first perform semantic segmentation of the original point cloud in a 3D scene, and then perform geometric vector extraction. Large-scale, richly annotated three-dimensional urban road scene lidar point cloud data is crucial for understanding complex roads and urban scenes, and can provide the main data source for important requirements such as autonomous driving map construction, three-dimensional model reconstruction, and smart cities. During actual point cloud semantic segmentation, due to the high computational complexity of full-frame three-dimensional calculations, the AI inference speed is slow. For urban road traffic components, for component categories that are continuous, regular, have obvious edge contours, and high-contrast textures, the ground object features are more obvious after being processed by the point cloud projection algorithm. Therefore, in order to improve the effect of the point cloud semantic segmentation network and enhance the AI inference speed, it is necessary to study the point cloud projection algorithm to enhance the significant features of ground objects. In recent years, point cloud data projection technology has made remarkable progress in the field of three-dimensional data processing, and many projection methods such as multi-view projection, bird's-eye view projection, cylindrical projection, and spherical projection have emerged, providing powerful tools for the understanding and reconstruction of complex scenes.

[0003] The multi-view projection technology effectively solves the perspective limitation problem of traditional single-view methods by capturing data from multiple angles and projecting this data onto a two-dimensional plane. However, each view only shows the specific direction information of the ground object. During the projection process, there is partial occlusion during the perspective switch, which affects the acquisition of complete depth information and structural details.

[0004] The bird's-eye view has the advantages of strong realism, being able to better show the overall view and spatial relationship of large buildings or landscapes, as well as multi-view fusion, retaining geometric structure and semantic density. However, due to its aerial view from a high altitude, a large amount of characteristic information, especially depth information, is lost, and there is also mutual occlusion between ground objects in the vertical perspective.

[0005] The advantage of cylindrical projection is that it can retain the angular information of the point cloud, which is very beneficial for analyzing the surrounding structure or rotational symmetry of objects. However, it cannot show ground objects with vertical structures well, and there are distortion situations for ground objects at the edge of the cylinder.

[0006] Spherical projection technology aims to maintain the integrity of information and provide a full - field - of - view perspective by projecting point - cloud data onto a sphere, reducing the impact of viewing - angle limitations. However, when dealing with dense point clouds, problems such as projection distortion and uneven information density often occur in the edge regions, resulting in data distortion and loss of feature information. These limitations restrict the application effect of spherical projection in processing high - density point - cloud data.

[0007] In summary, although certain progress has been made in existing point - cloud data projection technologies, key issues such as information loss, sensitivity to viewing - angle selection, occlusion handling, and projection distortion that arise under the demand for processing large - scale dense point clouds still need to be addressed. Summary of the Invention

[0008] To address the deficiencies of the existing technology, achieve enhanced retention of ground - object features in dense point clouds, alleviate the deformation of ground objects during the projection process, and reduce the occlusion between ground objects, the present invention adopts the following technical solutions:

[0009] A dynamic projection method for point - cloud data, comprising the following steps:

[0010] Step 1: Calculate the road slope from the acquired original point cloud, segment the original point cloud according to the road slope to obtain a set of partitioned point clouds, and calculate the road - density centerlines of each piece of partitioned point cloud;

[0011] Step 2: Grid each piece of partitioned point cloud to obtain a set of grid point clouds, and calculate the density center of each grid point cloud, the nearest point of the density center to the road - density centerline, and the tangent direction of the road - density centerline at the nearest point. Adaptively establish a dynamic three - dimensional coordinate system with the density center and the tangent direction; transform each grid point cloud into the corresponding dynamic three - dimensional coordinate system to obtain new grid point clouds, and obtain the horizontal circumscribed cylinders of the new grid point clouds;

[0012] Step 3: Based on the horizontal circumscribed cylinder, project the point cloud inside the cylinder onto the side surface of the cylinder;

[0013] Step 4: Unroll the side surface of the cylinder and map the projected points on the side surface onto a projection map.

[0014] Furthermore, in Step 1, calculate the slope of the road point cloud, and use the point with a larger slope as the initial dividing point of the original point cloud. If the initial dividing points are too dense, take the point with the largest slope (local maximum) as the dividing point. If they are too sparse, perform interpolation calculation on the initial dividing points, and use the calculated interpolation points as the dividing points. If the slope change is not obvious, do not consider the slope and directly divide evenly, as much as possible ensuring that the lengths of the partitioned point clouds obtained after dividing the original point cloud are roughly the same.

[0015] Further, the dynamic three-dimensional coordinate system established in step 2 is based on the three-dimensional coordinates of the grid point cloud. Taking the density center as the coordinate origin and the vector in the tangent direction as the new Z-axis coordinate, a new three-dimensional coordinate system is established. Through the transformation matrix from the original three-dimensional coordinate system to the new three-dimensional coordinate system, the grid point cloud is transformed into the new three-dimensional coordinate system. Based on the dynamic coordinate axes, it is possible to adaptively adjust for each piece of point cloud, find a cylinder with a suitable radius, side height, and center position of the circle, and perform a more appropriate projection on each piece of point cloud.

[0016] Further, in step 2, an external circumscribed cylinder is made for the new grid point cloud. The axis of the cylinder is collinear with the tangent direction, and the center point of the cylinder is the new density center after transformation. Calculate the projection length of the new three-dimensional coordinates of the grid point cloud in the tangent direction. Based on the projection length difference, obtain the height of the external circumscribed cylinder. Calculate the radius of the bottom surface of the external circumscribed cylinder by the difference between the new three-dimensional coordinates and the new density center.

[0017] Further, the radius of the bottom surface of the external circumscribed cylinder is the maximum value found in the absolute value of the difference between the new three-dimensional coordinates and the new density center.

[0018] Further, in step 3, for each point in the new grid point cloud, find the circular plane where it is located in the external circumscribed cylinder. Calculate the arctangent through the difference between the x coordinate of the new grid point in the circular plane and the x coordinate of the new density center point, and the difference between the y coordinate and the y coordinate of the new density center point to obtain the corresponding central angle. Calculate the projection points of the x and y coordinates on the circumference through the central angle and the radius of the bottom surface of the cylinder. Combine the difference between the z coordinate of the new grid point cloud and the z coordinate of the new density center point to obtain the projection of the grid point cloud on the side of the cylinder. When calculating the projection points on the side of the cylinder, by finding the circle parallel to the bottom surface of the cylinder corresponding to each 3D point and then calculating the central angle, for points at the same height, the closer to the central axis, the larger the central angle and the farther the position on the side of the cylinder. This can make the objects behind the occlusion position appear more on the projection map, with less occlusion between objects, and compared with the projection method that diverges outward from the center point or other specific points, the deformation of the objects is also less.

[0019] Further, in step 4, each pixel point of the projection carries point cloud information including the original coordinates and reflection intensity.

[0020] Further, the method further includes step 5 of inputting the projected data, combined with the original point cloud and voxelized data, into a multi-modal point cloud semantic segmentation network to obtain the ground object segmentation result.

[0021] A dynamic projection method for vehicle-mounted lidar point cloud data calculates the road slope from the original point cloud obtained by vehicle-mounted lidar, and performs projection using the dynamic projection method for point cloud data described above.

[0022] A dynamic projection system for vehicle-mounted lidar point cloud data includes a segmentation module, a gridification module, a three-dimensional coordinate system construction module, a cylinder generation module, a projection module, and a mapping module. According to the dynamic projection method for point cloud data described above, the original point cloud obtained by vehicle-mounted lidar is segmented, gridified, adaptively establishes a dynamic three-dimensional coordinate system, generates a horizontal circumscribed cylinder of the grid point cloud, projects the point cloud onto the side of the cylinder, and finally maps the projection points on the side to the projection map.

[0023] The advantages and beneficial effects of the present invention are as follows:

[0024] The dynamic projection method and system for vehicle-mounted lidar point cloud data of the present invention can better enhance the retention of ground object features in large-scale 3D dense point cloud data, greatly alleviate the deformation of ground objects during the projection process, and minimize the confusion in the semantic segmentation task; there are more projection performances for the ground objects behind the occlusion position, less occlusion between ground objects, and improved subsequent segmentation effects; when the generated projection map is used for subsequent deep learning tasks, the inference speed is better than that of networks based on points and voxels. Description of the Drawings

[0025] Figure 1 is the flowchart of the method in the embodiment of the present invention.

[0026] Figure 2 is the grid point cloud map before conversion in the embodiment of the present invention.

[0027] Figure 3 is the point cloud map after conversion to the new coordinate system in the embodiment of the present invention.

[0028] Figure 4a is the point cloud map after dynamic horizontal cylinder projection in the embodiment of the present invention (viewpoint one).

[0029] Figure 4b is the point cloud map after dynamic horizontal cylinder projection in the embodiment of the present invention (viewpoint two).

[0030] Figure 5 is the projection map after unfolding the dynamic horizontal cylinder projection in the embodiment of the present invention. Detailed Embodiments

[0031] The following details the specific embodiments of the present invention in conjunction with the accompanying drawings. It should be understood that the specific embodiments described herein are only for the purpose of illustrating and explaining the present invention, and are not used to limit the present invention.

[0032] In the prior art, there is no projection method for three-dimensional dense point clouds. Applying existing projection methods directly to dense point clouds will cause serious data distortion, specifically including: 1) As the latitude increases, the deformation of the object becomes more serious and the feature loss is greater; 2) There is serious occlusion when existing projection algorithms are applied to dense point clouds.

[0033] In view of the above problems, the present invention proposes a dynamic projection method for vehicle-mounted laser point cloud data. As Figure 1 shown, the point cloud data is divided into four dimensions, including large-scale point clouds that have not undergone any segmentation operations (hereinafter referred to as original point clouds), point clouds obtained by partitioning the original point clouds according to the approximate slope (hereinafter referred to as partitioned point clouds), point clouds obtained by subjecting the partitioned point clouds to a 10m * 10m grid processing (grid point clouds), and all individual point clouds in the grid point clouds (hereinafter referred to as 3D points). The dynamic projection method specifically includes the following steps:

[0034] Step 1: For vehicle-mounted dense original point cloud data, segment it into partitioned point clouds according to the approximate slope of the road; for each partitioned point cloud A i calculate its road density center line ; The segmentation of the partitioned point cloud includes the following steps:

[0035] Step 1.1: Calculate the slope of each point

[0036] Let the point cloud coordinate list be (N represents the number of point cloud coordinates, arranged in ascending order, and only and and are considered when calculating the slope). The central difference method is used to calculate the slope of each point. For internal points:

[0037]

[0038] Step 1.2: Check whether the slope change is obvious

[0039] Set the slope change threshold , check the slopes of all points. If the slope change is less than the threshold, that is, the difference between the maximum slope and the minimum slope , then directly divide according to the equal division principle and directly go to Step 1.6. If the slope change is greater than , then execute Step 1.3; The equal division principle division formula is as follows:

[0040]

[0041] Step 1.3: Selection of candidate division points

[0042] Select local maxima as candidate division points: that is, local slope maximum points that satisfy and .

[0043] Step 1.4: Process dense division points

[0044] Set the ideal partition length L and the dense threshold , traverse the candidate division points. If the distance between adjacent points , then only keep the point with the maximum slope in the interval , and remove the rest.

[0045] Step 1.5: Process sparse division points

[0046] Set the sparse threshold , traverse the candidate division points. If the distance between adjacent points , then interpolate according to the following rules:

[0047] Calculation of the number of interpolation points:

[0048]

[0049] Linear interpolation coordinates:

[0050]

[0051] Step 1.6: Generate the final partition

[0052] Segment the original point cloud into according to the division points.

[0053] Step 2: For A i Grid it into a 10m * 10m grid point cloud S according to the point cloud coordinate positions j , and its spatial coordinates P k (x, y, z); For the grid point cloud S after meshing j calculate its density center O j , the distance L j from O i The tangent direction vector D i of the nearest point on L i , take O j as the coordinate origin, and take D i as the new z-axis to establish a new coordinate system. The transformation matrix from the original coordinate system to the new coordinate system is denoted as T i ; Transform the grid point cloud S in the original coordinate system j through the transformation matrix T i to the new coordinate system, denoted as , and the spatial coordinates of the 3D points in the transformed grid point cloud are denoted as :

[0054]

[0055] For Construct an external cylinder. The axis of the cylinder is collinear with the unit direction vector, and the center point of the cylinder is O j Through the transformation matrix T i After transformation, it is denoted as . Calculate The projection length proj in the direction of the direction vector D i : k :

[0056]

[0057] The height H of the external cylinder is obtained through calculation j :

[0058]

[0059] The radius R of the bottom surface of the external cylinder j And other parameters:

[0060]

[0061] In another embodiment, first establish a new coordinate system, transform the grid point cloud to the new coordinate system. In actual operation, a cylinder is also established in the original coordinate system, and a transformation matrix is established before projection to transform the grid point cloud and the cylinder to the new coordinate system.

[0062] Step 3: For each 3D point in the grid point cloud, find the circular plane where it is located in the external cylinder, and calculate the central angle θ corresponding to each point k :

[0063]

[0064] Through the radius R j And the central angle θ k The projection points corresponding to each point on the circumference are obtained through calculation:

[0065]

[0066]

[0067]

[0068] Step 4: Project the point cloud projection points located on the side of the cylinder obtained in Step 3, expand them on the side according to the cylinder parameters obtained in Step 2, and then map them proportionally onto a projection diagram with a specified width and height. Each pixel point carries effective information such as the original coordinates and reflection intensity of the point cloud;

[0069] Step 5: Input the projected data, combined with the original point cloud and the voxelized data, into a multi-modal point cloud semantic segmentation network to obtain the ground object segmentation result.

[0070] In one embodiment of the present invention, the segmentation of the partitioned point cloud in step 1 is as follows:

[0071] The point cloud coordinates are , , then the corresponding slope is obtained:

[0072]

[0073] For boundary points (when i = 1 and i = N):

[0074]

[0075] The corresponding boundary point coordinates are , , then the corresponding slope is obtained:

[0076]

[0077] For those with the difference between the maximum slope and the minimum slope less than the slope change threshold, the point cloud data is divided according to the equal division principle:

[0078]

[0079] Set the ideal partition length L - 300m, and set the corresponding density threshold to , traverse the candidate division points. If the distance between adjacent points , then only the point with the maximum slope is retained in the interval , and the rest are removed.

[0080] Set the sparse threshold , traverse the candidate division points. If the distance between adjacent points , then calculate the number of inserted points as follows:

[0081]

[0082] Calculate the linear interpolation coordinates:

[0083]

[0084] Finally, segment the original point cloud according to the division points.

[0085] In step 2, each partitioned point cloud is meshed into a 10m * 10m grid point cloud according to the point cloud coordinate position. As Figure 2 shown, its spatial coordinate P k (x, y, z), when x is the largest, P max(x,y,z) = (6.27459999e+05, 4.84280703e+06, 1.52117004e+02), when x is the smallest, P min (x,y,z) = (6.27450000e+05, 4.84280536e+06, 1.38839005e+02); for the point cloud of the meshed points, calculate the density center O = (6.27456417e+05, 4.84280574e+06, 1.42987188e+02). The tangent direction vector of the point on the road density center line that is closest to the density center of the road density center line is D = (-0.3001755, 0.9536763, 0.01990438). Taking the density center as the coordinate origin and the tangent direction vector as the new z-axis, establish a new coordinate system, as Figure 3 shown, the transformation matrix from the original coordinate system to the new coordinate system is:

[0086]

[0087] The spatial coordinates of the point cloud after transformation become:

[0088]

[0089] Then The projection length on the direction vector is ;

[0090] The height of the circumscribed cylinder is ;

[0091] The radius of the bottom surface of the circumscribed cylinder is:

[0092]

[0093] In step 3, for each point in the grid point cloud, find the circular plane where it is located in the circumscribed cylinder to obtain the central angle:

[0094]

[0095] Calculate the dynamic horizontal cylinder projection points through the central angle and the radius of the bottom surface of the circumscribed cylinder,

[0096]

[0097]

[0098]

[0099]

[0100] The projected point cloud, as Figure 4a 、Figure 4b as shown

[0101] Unfold the projected points of the point cloud on the side of the cylinder, and then map them proportionally onto a projected image with a specified width and height to obtain the projected image after unfolding and mapping, as Figure 5 shown

[0102] Applying existing point cloud projection algorithms directly to dense point clouds will cause serious deformation of ground objects and loss of feature information. However, the present invention projects dense point clouds, making up for the defects of current projection methods. The present invention follows the principle that points closer to the lidar have a higher density than those farther away. By detecting areas with high point cloud density, the lidar scanning trajectory is identified, and after aligning the projection direction with the scanning direction, 2D projection of dense point clouds is performed, better restoring ground objects in the real scene, greatly reducing deformation, and enhancing the retention of ground object features.

[0103] A dynamic projection system for vehicle-mounted lidar point cloud data includes a segmentation module, a meshing module, a three-dimensional coordinate system construction module, a cylinder generation module, a projection module, and a mapping module. According to the dynamic projection method of point cloud data described above, the original point cloud obtained by vehicle-mounted lidar is segmented, meshed, adaptively establish a dynamic three-dimensional coordinate system, generate a horizontal circumscribed cylinder of the grid point cloud, project the point cloud onto the side of the cylinder, and finally map the projected points on the side onto a projected image.

[0104] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A dynamic projection method for point cloud data, characterized in that It includes the following steps: Step 1: Calculate the road slope from the acquired original point cloud, segment the original point cloud according to the road slope to obtain a set of partitioned point clouds, and calculate the road density centerlines of each piece of partitioned point cloud; Step 2: Grid each piece of partitioned point cloud to obtain a set of grid point clouds, and calculate the density center of each grid point cloud, the nearest point of the density center to the road density centerline, and the tangent direction of the road density centerline at the nearest point. Adaptively establish a dynamic three-dimensional coordinate system with the density center and the tangent direction; Transform each grid point cloud into the corresponding dynamic three-dimensional coordinate system to obtain a new grid point cloud, and obtain the horizontal circumscribed cylinder of the new grid point cloud; Make a circumscribed cylinder for the new grid point cloud, the axis of the cylinder is collinear with the tangent direction, the center point of the cylinder is the new density center after transformation, calculate the projection length of the new three-dimensional coordinates of the grid point cloud in the tangent direction, obtain the height of the circumscribed cylinder based on the projection length difference, and calculate the bottom radius of the circumscribed cylinder by the difference between the new three-dimensional coordinates and the new density center; Step 3: Based on the horizontal circumscribed cylinder, project the point cloud inside the cylinder onto the side surface of the cylinder; For each point in the new grid point cloud, find the circular plane where it is located in the circumscribed cylinder, calculate the arctangent to obtain the corresponding central angle through the difference between the x coordinate of the new grid point in the circular plane and the x coordinate of the new density center point, and the difference between the y coordinate and the y coordinate of the new density center point. Calculate the projection points of the x and y coordinates on the circumference through the central angle and the bottom radius of the cylinder, and combine the difference between the z coordinate of the new grid point cloud and the z coordinate of the new density center point to obtain the projection of the grid point cloud on the side surface of the cylinder; Step 4: Unroll the side surface of the cylinder and map the projection points on the side surface to the projection map.

2. The dynamic projection method for point cloud data according to claim 1, wherein: In the said Step 1, calculate the slope of the road point cloud, and use the point with the larger slope as the preliminary division point. If the preliminary division points are dense, take the point with the largest slope among them as the division point. If the preliminary division points are sparse, perform interpolation calculation on the preliminary division points, and use the calculated interpolation points as the division points. If the slope change is small, directly divide evenly.

3. A dynamic projection method for point cloud data according to claim 1, characterized in that: The dynamic three-dimensional coordinate system established in the said Step 2 is based on the three-dimensional coordinates of the grid point cloud. Take the density center as the coordinate origin, use the vector of the tangent direction as the new Z-axis coordinate to establish a new three-dimensional coordinate system, and transform the grid point cloud into the new three-dimensional coordinate system through the transformation matrix from the original three-dimensional coordinate system to the new three-dimensional coordinate system.

4. A dynamic projection method for point cloud data according to claim 1, characterized in that: The bottom radius of the circumscribed cylinder is the maximum value found in the absolute value of the difference between the new three-dimensional coordinates and the new density center.

5. The dynamic projection method of point cloud data according to claim 1, characterized in that: In the said Step 4, each pixel point of the projection carries point cloud information including the original coordinates and reflection intensity.

6. The dynamic projection method of point cloud data according to claim 1, characterized in that: The method further includes Step 5, input the projected data, combined with the original point cloud and voxelized data, into a multi-modal point cloud semantic segmentation network to obtain the ground object segmentation result.

7. A dynamic projection method for vehicle-mounted laser point cloud data, characterized in that: Calculate the road slope from the original point cloud acquired by vehicle-mounted laser, and perform projection using a dynamic projection method for point cloud data according to any one of claims 1-6.

8. A dynamic projection system for vehicle-mounted lidar point cloud data, comprising a segmentation module, a gridification module, a three-dimensional coordinate system construction module, a cylinder generation module, a projection module, and a mapping module, characterized in that: A dynamic projection method for point cloud data according to claim 1, which respectively segments and grids the original point cloud obtained by vehicle-mounted laser, adaptively establishes a dynamic three-dimensional coordinate system, generates a horizontal circumscribed cylinder of the grid point cloud, projects the point cloud onto the side surface of the cylinder, and finally maps the projection points on the side surface onto the projection map.

Images