A method and system for processing lidar point cloud data

Abstract

This invention provides a method and system for processing lidar point cloud data. The method includes: acquiring point clouds of a target object scanned by the lidar within a target detection area; selecting point clouds containing the target object by setting a Region of Interest (ROI); extracting and matching point cloud features at the scale of a single lidar scan line to obtain candidate point clouds; extracting and matching point cloud features at the overall point cloud scale to obtain denoised and refined candidate point clouds; and performing RANSAC operations on the denoised and refined candidate point clouds to filter out falsely detected point clouds, thus obtaining the target object point cloud. This invention can improve the timeliness of target object detection and is suitable for operation on automotive-grade chips.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving technology, and in particular to a method and system for processing lidar point cloud data. Background Technology

[0002] In the field of autonomous driving, satellite positioning is commonly used to determine the location of autonomous vehicles. When an autonomous vehicle is driving on a road with weak or no satellite positioning signal, its positioning will be affected, which will limit the operating scenarios of the autonomous vehicle.

[0003] Currently, to address the aforementioned issues, radar devices are used for assisted localization of autonomous vehicles. This involves acquiring all point cloud data scanned by the radar during vehicle movement and then applying deep learning algorithms to this data to determine the location information of landmarks, thereby achieving vehicle localization. However, using deep learning algorithms for landmark detection places high demands on chip computing power, while existing automotive-grade chips have relatively weak computing capabilities, severely impacting the timeliness of landmark detection and posing a potential threat to the safe operation of autonomous vehicles. Summary of the Invention

[0004] The purpose of this invention is to provide a method and system for processing lidar point cloud data, which can improve the timeliness of marker detection and is suitable for marker detection operations on automotive-grade chips. The specific technical solution is as follows:

[0005] This invention provides a method for processing lidar point cloud data, comprising:

[0006] Acquire the point cloud of the target marker scanned by the lidar within the target detection area, and select the point cloud containing the target marker by setting the ROI region;

[0007] Point cloud features are extracted and matched at the scale of a single laser scan line for each scan line of the lidar to obtain candidate point clouds;

[0008] The candidate point cloud is subjected to point cloud feature extraction and matching at the overall point cloud scale to obtain a denoised and refined candidate point cloud.

[0009] The RANSAC operation is performed on the denoised and refined candidate point cloud to filter out the falsely detected point clouds and obtain the marker point cloud.

[0010] Optional, also includes:

[0011] Using the point cloud of the marker, the feature parameters of the marker are calculated, wherein the point cloud of the marker is a three-dimensional point cloud.

[0012] Optionally, calculating the feature parameters of the markers using the marker point cloud includes:

[0013] The point cloud of the markers is projected onto the two-dimensional coordinate system of the lidar to obtain a two-dimensional point cloud. The two-dimensional coordinate system of the lidar is the XY coordinate system.

[0014] The feature parameters of the marker are obtained by least squares fitting in the two-dimensional coordinate system of the lidar. The feature parameters of the marker include the slope along the distribution direction of the two-dimensional point cloud in the lidar point cloud two-dimensional coordinate system and the distance from the origin of the vehicle coordinate system in the normal direction.

[0015] Optionally, the step of selecting a point cloud containing the marker to be inspected by setting the ROI region includes:

[0016] The lateral range of the Area of ​​Interest (ROI) is obtained by measuring the lateral distance between road boundary markers and the vehicle's autonomous driving route. The lateral range of the ROI is defined as the range along the Y-axis of the LiDAR's three-dimensional coordinate system.

[0017] The longitudinal range of the Region of Interest (ROI) is obtained by measuring the detection range of the marker using the lidar, where the longitudinal range of the ROI is the range along the X direction of the lidar's three-dimensional coordinate system.

[0018] The ROI height range is obtained by scanning the pitch angle range with a lidar scanner. The ROI height range is the range along the Z-axis of the lidar's three-dimensional coordinate system.

[0019] The point cloud containing the marker to be inspected is selected by the ROI horizontal range, the ROI vertical range, and the ROI height range.

[0020] Optionally, the step of extracting and matching point cloud features at the scale of a single laser scan line for each scan line of the lidar to obtain candidate point clouds includes:

[0021] Based on the characteristics of the scan lines acquired by the lidar, and combined with the installation position and pose information of the lidar, the scan line range corresponding to the target object is determined.

[0022] The point cloud is filtered within the scan line range, and Euclidean clustering is performed on the point cloud on each scan line after filtering to obtain candidate point cloud clusters.

[0023] The features of the candidate point cloud clusters are extracted and matched with the features of the markers to obtain the candidate point cloud.

[0024] Optionally, determining the scan line range corresponding to the target object based on the features of the scan line acquired by the lidar, combined with the installation position and pose information of the lidar, includes:

[0025] Calculate the angular range between the laser radar scanning the target object in a direction perpendicular to the laser radar scanning direction, where the angular range is [θ]. min ,θ max ];

[0026] θ is determined according to the following formula. min :

[0027]

[0028] θ is determined according to the following formula. max :

[0029]

[0030] In the formula, θ min h is the angle between the line connecting the lowest point of the lidar and the target object and the ground. min h is the minimum distance between the marker to be inspected and the ground. L θ is the height of the lidar above the ground. max h is the angle between the line connecting the highest point of the lidar and the target object and the ground. max The distance between the target object and the ground is the maximum value, where d is the distance between the lidar and the target object, and d∈[d]. min ,d max ], d min d represents the closest distance between the lidar and the target object within the lidar's detection range. max The furthest distance between the lidar and the target object within the lidar's detection range;

[0031] The scan line range corresponding to the marker to be inspected is determined based on the included angle range.

[0032] Optionally, the step of extracting features from the candidate point cloud clusters and matching them with marker features to obtain the candidate point cloud includes:

[0033] Features of the candidate point cloud clusters are extracted and matched with the features of the marker. Multiple candidate point cloud clusters that match the same marker on a single scan line are merged by interpolation to obtain the candidate point cloud.

[0034] Optionally, the step of extracting and matching point cloud features at the overall point cloud scale to obtain denoised and refined candidate point clouds includes:

[0035] By utilizing the structural features of the markers and / or the positional features of adjacent markers, candidate point clouds whose point cloud feature values ​​are within a first preset range are selected from the candidate point clouds to obtain the denoised and refined candidate point clouds. The structural features of the markers include at least one of the marker length, marker height, and marker curvature, and the positional features of adjacent markers include the spacing between adjacent markers.

[0036] Optionally, the step of extracting and matching point cloud features at the overall point cloud scale to obtain denoised and refined candidate point clouds includes:

[0037] Using the distance between the centroid of the candidate point cloud and the vehicle, candidate point clouds with distance values ​​within a second preset range are selected from the candidate point clouds to obtain the denoised and refined candidate point clouds.

[0038] The present invention also provides a lidar point cloud data processing system, comprising:

[0039] The first point cloud filtering module is used to acquire the point cloud on the target marker scanned by the lidar within the target detection area, and select the point cloud containing the target marker by setting the ROI area.

[0040] The second point cloud filtering module is used to extract and match point cloud features at the scale of a single laser scan line for each scan line of the lidar to obtain candidate point clouds.

[0041] The third point cloud screening module is used to extract and match point cloud features at the overall scale of the candidate point cloud to obtain denoised and refined candidate point clouds.

[0042] The fourth point cloud filtering module is used to perform RANSAC operation on the denoised and refined candidate point cloud to filter out the falsely detected point cloud and obtain the marker point cloud.

[0043] This invention provides a method and system for processing lidar point cloud data. The method involves acquiring point clouds of a target object scanned by the lidar within a target detection area, selecting point clouds containing the target object by setting a Region of Interest (ROI), extracting and matching point cloud features at the scale of each lidar scan line to obtain candidate point clouds, extracting and matching point cloud features at the overall point cloud scale to obtain denoised and refined candidate point clouds, and performing RANSAC operations on the denoised and refined candidate point clouds to filter out falsely detected point clouds, thus obtaining the target object point cloud. This invention can quickly filter a large number of point clouds to obtain the target object point cloud, and the target object location information can be directly obtained from the target object point cloud, thereby improving the timeliness of target object detection and making it suitable for target object detection operations on automotive-grade chips.

[0044] Of course, any product or method implementing the present invention does not necessarily need to achieve all of the advantages described above at the same time. Attached Figure Description

[0045] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0046] Figure 1 A flowchart of a lidar point cloud data processing method provided in an embodiment of the present invention;

[0047] Figure 2 A schematic diagram of an autonomous vehicle provided in an embodiment of the present invention;

[0048] Figure 3 This is a schematic diagram of the region of interest provided in an embodiment of the present invention;

[0049] Figure 4 (a) is a schematic diagram showing the distance between the lidar and the marker;

[0050] Figure 4 (b) is a schematic diagram showing the angle between the lidar and the marker;

[0051] Figure 5 A schematic diagram of point cloud clustering on a single lidar scanning line provided in an embodiment of the present invention;

[0052] Figure 6 A flowchart of another lidar point cloud data processing method provided in an embodiment of the present invention;

[0053] Figure 7 This is a schematic diagram illustrating the positional relationship between the marker and the vehicle provided in an embodiment of the present invention;

[0054] Figure 8 A structural diagram of a lidar point cloud data processing system provided in an embodiment of the present invention;

[0055] Figure 9 Another lidar point cloud data processing system structure diagram provided in this embodiment of the invention;

[0056] Figure 10 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0057] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0058] In the field of autonomous driving, markers are typically placed on road sections with weak or no satellite positioning signals. These markers assist in the positioning of autonomous vehicles. The coordinates of the markers in the map coordinate system are usually known. By obtaining the coordinates of the markers in the vehicle's coordinate system, coordinate transformation can be used to determine the vehicle's coordinates in the map coordinate system, thereby achieving the positioning of the autonomous vehicle. Optionally, the markers can be poles, bollards, guardrails, etc.

[0059] The coordinates of a marker in the vehicle coordinate system can be determined using a binocular vision method with cameras mounted on the vehicle. However, camera detection is prone to failure in poor lighting conditions (such as at night) or in rainy or snowy weather, which limits the operating time or scenarios for autonomous vehicles. Compared to using cameras for marker localization, LiDAR is less affected by lighting and weather conditions, ensuring that the operating time or scenarios for autonomous vehicles are not limited. Therefore, this invention employs LiDAR to detect markers to facilitate the positioning of autonomous vehicles.

[0060] When using LiDAR for assisted localization of autonomous vehicles, LiDAR can scan all point cloud data. Detecting landmark point clouds from all point cloud data requires high chip computing power, but existing automotive-grade chips have weak computing power, which seriously affects the timeliness of landmark detection. This will pose a hidden danger to the safe operation of autonomous vehicles.

[0061] Based on this, the present invention provides a method for processing lidar point cloud data, which can be adapted to the operation of marker detection on automotive-grade chips. For example... Figure 1 As shown, the method includes:

[0062] Step 101: Obtain the point cloud of the target object scanned by the LiDAR within the target detection area, and select the point cloud containing the target object by setting the ROI region.

[0063] LiDAR can sense the environment around a vehicle. To extend its scanning range, LiDAR can be mounted on either side of the front of the vehicle. For example... Figure 2 As shown, Figure 2This is a schematic diagram of an autonomous vehicle. The origin of the vehicle coordinate system can be the intersection of the vehicle's centerline and the straight line where the front of the vehicle is located. The X-axis of this vehicle coordinate system is parallel to the vehicle's longitudinal centerline, the Y-axis is parallel to the straight line where the front of the vehicle is located, and the Z-axis is perpendicular to the XOY plane. The two LiDARs 2 located on both sides of the front of the vehicle 1 are represented by solid black circles. Optionally, these LiDARs can be mechanical LiDARs.

[0064] The target detection area can contain objects to be detected. While the vehicle is moving, the LiDAR installed on the vehicle can scan the point cloud of the objects to be detected within the target detection area. To reduce computation time and detection interference, a Region of Interest (ROI) can be set to select the point cloud containing the objects to be detected. Optionally, the ROI can be a three-dimensional rectangle.

[0065] As an optional implementation, the method for selecting a point cloud containing the target marker by setting the ROI region includes: obtaining the lateral range of the ROI by the lateral distance between the road boundary marker and the vehicle's autonomous driving route, wherein the lateral range of the ROI is the range in the Y direction of the LiDAR three-dimensional coordinate system; obtaining the longitudinal range of the ROI by the detection range of the marker by the LiDAR, wherein the longitudinal range of the ROI is the range in the X direction of the LiDAR three-dimensional coordinate system; obtaining the height range of the ROI by the range of changes in the LiDAR scanning pitch angle, wherein the height range of the ROI is the range in the Z direction of the LiDAR three-dimensional coordinate system; and selecting a point cloud containing the target marker by using the lateral range, longitudinal range, and height range of the ROI.

[0066] In this implementation, road boundary markers can be one or more of the following: stone blocks, poles, and guardrails. When the vehicle travels along the autonomous driving route, there is a lateral distance between the vehicle's longitudinal centerline and the road boundary markers. When determining the lateral range of the Area of ​​Interest (ROI), the upper limit of the lateral range can be obtained by adding a first preset distance to this lateral distance, or the lower limit can be obtained by subtracting a second preset distance from this lateral distance. The first preset distance can be the same as or different from the second preset distance. When determining the longitudinal range of the ROI, it can be determined based on the detection range of the LiDAR. Within the LiDAR detection range, a range containing at least one complete marker can be selected as the longitudinal range of the ROI. Optionally, the longitudinal range of the ROI can contain two complete markers. Different LiDARs may have different scanning pitch angles; the range of variation in the LiDAR scanning pitch angle can be used as the height range of the ROI.

[0067] like Figure 3 As shown, Figure 3As a schematic diagram of the region of interest, the two LiDAR sensors 2 on either side of the front of vehicle 1 are represented by solid black circles. The dashed boxes on either side of vehicle 1 represent the regions of interest 3, and the solid rectangular areas within each region of interest 3 represent two objects to be inspected 4. From Figure 3 As can be seen, each region of interest can contain two targets to be detected. The X-axis of the LiDAR 3D coordinate system can be the same as the X-axis of the vehicle coordinate system, the Y-axis of the LiDAR 3D coordinate system can be the same as the Y-axis of the vehicle coordinate system, the Z-axis of the LiDAR 3D coordinate system can be the same as the Z-axis of the vehicle coordinate system, the origins of the LiDAR coordinate system and the vehicle coordinate system can both be located on the Y-axis of the vehicle coordinate system, and coordinate system transformation can be performed between the LiDAR coordinate system and the vehicle coordinate system.

[0068] Step 102: Extract and match point cloud features at the scale of a single laser scan line for each scan line of the lidar to obtain candidate point clouds.

[0069] LiDAR can acquire point cloud data of the environment around a vehicle. Due to the complexity of the environment around a vehicle, the point cloud acquired by LiDAR will include other point clouds besides the point clouds of landmarks. In order to avoid the long computation time caused by point cloud clustering of a large number of point clouds, this invention extracts and matches point cloud features at the scale of a single laser scan line for each scan line of the LiDAR, thereby obtaining candidate point clouds.

[0070] As an optional implementation, point cloud features are extracted and matched at the scale of a single laser scan line for each scan line of the lidar to obtain candidate point clouds. This includes: determining the scan line range corresponding to the target object based on the features of the scan lines acquired by the lidar, combined with the installation position and pose information of the lidar; filtering the point clouds within the scan line range, and performing Euclidean clustering on the point clouds on each scan line after filtering to obtain candidate point cloud clusters; extracting the features of the candidate point cloud clusters and matching them with the features of the target object to obtain candidate point clouds.

[0071] Optionally, the lidar can be a mechanical lidar with a radial scanning range of 360°. This mechanical lidar can emit multiple scan lines, the characteristics of which vary depending on the type and model of the lidar. The number of scan lines can be determined based on the lidar's elevation angle range and vertical angle (i.e., the elevation angle between two adjacent scan lines). The vertical angle between two adjacent scan lines can be fixed (e.g., 2°), and the scan lines are arranged equidistantly from top to bottom on the same plane.

[0072] Different scan lines can acquire point clouds at different heights. To determine the scan line where a marker is located, the scan line range corresponding to the marker to be inspected can be determined by combining the installation position and pose information of the LiDAR. Given the installation height and pose of the LiDAR, the height of each laser scan line at any distance within the LiDAR's range can be calculated. This allows us to extrapolate the longitudinal range of the scan lines that the LiDAR might scan for markers along the vehicle's autonomous driving route. For each type of road boundary marker, a set of scan line ranges corresponds to it, serving as a filtering threshold for online detection.

[0073] The method for determining the scan line range corresponding to the target object based on the characteristics of the scan line acquired by the lidar, combined with the lidar's installation position and pose information, can be found in [reference needed]. Figure 4 , Figure 4 (a) is a schematic diagram showing the distance between the lidar and the marker. Figure 4 (b) is a schematic diagram of the angle between the lidar and the target object to be inspected. Figure 4 (a) is a top view. Figure 4 (a) The arrow in the middle indicates the autonomous driving route of vehicle 1. LiDAR 2 is represented by a solid black circle. Within the detection range of the LiDAR, the closest distance between LiDAR 2 and the target marker 4 is d. min The farthest distance is d max ; Figure 4 (b) is the rear view. Figure 4 (b) The solid black circle represents lidar 2, and the solid rectangular bar represents the target object 4. The distance between lidar 2 and target object 4 is d, where d∈[d min ,d max The height of lidar 2 above the ground is h. L The minimum distance between the marker to be inspected and the ground is h. min The maximum distance between the marker to be inspected and the ground is h. max The angle between the line connecting the lidar and the lowest point of the marker and the ground is θ. min The angle between the line connecting the lidar and the highest point of the marker and the ground is θ. max .

[0074] Optionally, based on the characteristics of the scan line acquired by the lidar, and combined with the lidar's installation position and pose information, the scan line range corresponding to the target object is determined. This includes: calculating the angle range between the lidar scanning the target object in a direction perpendicular to the lidar's scanning direction, and determining the scan line range corresponding to the target object based on this angle range. The lidar's scanning direction can be parallel to the ground, and the angle range is [θ]. min ,θ max ].

[0075] θ is determined according to the following formula.min :

[0076]

[0077] θ is determined according to the following formula. max :

[0078]

[0079] In the formula, θ min Let θ be the angle between the line connecting the lidar and the lowest point of the target object and the ground. max The angle between the line connecting the lidar and the highest point of the target object and the ground.

[0080] Optionally, the features of candidate point cloud clusters are extracted and matched with the features of the marker to obtain candidate point clouds, including: extracting the features of candidate point cloud clusters and matching them with the features of the marker, and merging multiple candidate point cloud clusters that match the same marker on a single scan line by interpolation to obtain candidate point clouds.

[0081] like Figure 5 As shown, this invention can utilize the scan line range obtained offline. For a certain type of marker, it can search within its corresponding scan line range and filter out those belonging to the scan line range [N] within the ROI region. min N max The point cloud of [ ]. After completing the above filtering, Euclidean clustering is performed on the point cloud along each scan line, such as Figure 5 The point cloud emitted by the Nth scan line from the LiDAR 2 can be clustered to obtain three candidate point cloud clusters, namely 5-1, 5-2, and 5-3. After obtaining the candidate point cloud clusters, features of the candidate point cloud clusters can be extracted, such as calculating the Euclidean distance between the two farthest points in each candidate point cloud cluster, which is used as the matching feature of each candidate point cloud cluster.

[0082] After obtaining the features to be matched for each candidate point cloud cluster on a single scan line, these features are matched with the offline-obtained landmark features to filter out point clouds belonging to a specific landmark. Landmark features include, but are not limited to: the width and height of the landmark, the spacing between adjacent landmarks, and straightness. For example, for a sign, the width of the sign can be used as a landmark feature, and the range of width within the horizontal width of the sign [0.4m, 1.0m] can be used as the range for matching. Figure 5 If the length of candidate point cloud cluster 5-2 falls within this range, then the point cloud contained in candidate point cloud cluster 5-2 will be considered as the candidate point cloud. For the scan line range [N]... min N max Points that match a certain marker feature can be merged into a single candidate point cloud.

[0083] Because the lidar used has a low resolution, the point cloud quality is usually not ideal when detecting distant markers, and the continuity of the point sequence on a single scan line is poor, such as... Figure 5 Middle scan line N min The two candidate point cloud clusters in the middle can be processed by interpolation to increase the number of candidate point clouds. This can improve the stability and accuracy of marker detection when the point cloud is sparse.

[0084] Step 103: Extract and match point cloud features at the overall point cloud scale to obtain denoised and refined candidate point clouds.

[0085] As an optional implementation, point cloud features are extracted and matched at the overall point cloud scale to obtain denoised and refined candidate point clouds. This includes: selecting candidate point clouds whose feature values ​​fall within a first preset range from the candidate point clouds using marker structural features and / or adjacent marker position features, thereby obtaining denoised and refined candidate point clouds. The marker structural features include at least one of marker length, marker height, and marker curvature, and the adjacent marker position features include the spacing between adjacent markers. Optionally, marker length, marker height, marker curvature, and the spacing between adjacent markers can all be obtained offline.

[0086] For example, for a stone pier object, a single stone pier will reflect at least a set of point clouds with straight line features and within a certain length range. At the same time, the distance between adjacent stone piers is also within a set of fixed threshold ranges. Through these threshold parameters, the laser point clouds reflected by the stone piers can be matched and identified.

[0087] As another optional implementation, point cloud feature extraction and matching are performed on the overall scale of the candidate point cloud to obtain a denoised and refined candidate point cloud, including: using the distance between the centroid of the candidate point cloud and the vehicle, selecting candidate point clouds whose distance values ​​are within a second preset range from the candidate point cloud to obtain a denoised and refined candidate point cloud.

[0088] For example, when detecting double rows of stone blocks on a road, by calculating the distance between the centroid of the stone block point cloud and the vehicle, and selecting candidate point clouds whose distance values ​​fall within a second preset range, the point cloud of the row of stone blocks closest to the vehicle can be filtered out, thus obtaining a denoised and refined candidate point cloud. Optionally, the distance between the centroid of the stone block point cloud and the vehicle can be calculated online.

[0089] Step 104: Perform RANSAC operation on the denoised and refined candidate point cloud to filter out the falsely detected point cloud and obtain the marker point cloud.

[0090] To extract more accurate feature parameters of markers (such as the distribution direction of markers, the distance between markers and vehicles, etc.), RANSAC operation can be performed on the denoised and refined candidate point cloud to filter out falsely detected point clouds and achieve point cloud denoising.

[0091] This invention also provides a method for processing lidar point cloud data, such as... Figure 6 As shown, the method includes:

[0092] Step 601: Obtain the point cloud of the target object scanned by the lidar within the target detection area, and select the point cloud containing the target object by setting the ROI region.

[0093] Among them, step 601 and Figure 1 Step 101 is similar and will not be repeated here.

[0094] Step 602: Extract and match point cloud features at the scale of a single laser scan line for each scan line of the lidar to obtain candidate point clouds.

[0095] Among them, step 602 and Figure 1 Step 102 is similar and will not be repeated here.

[0096] Step 603: Extract and match point cloud features at the overall point cloud scale to obtain denoised and refined candidate point clouds.

[0097] Among them, step 603 and Figure 1 Step 103 is similar and will not be repeated here.

[0098] Step 604: Perform RANSAC operation on the denoised and refined candidate point cloud to filter out the falsely detected point cloud and obtain the marker point cloud.

[0099] Among them, step 604 and Figure 1 Step 104 is similar and will not be repeated here.

[0100] Step 605: Calculate the feature parameters of the markers using the marker point cloud, where the marker point cloud is a three-dimensional point cloud.

[0101] As an optional implementation, the marker feature parameters are calculated using the marker point cloud, including: projecting the marker point cloud onto the two-dimensional coordinate system of the lidar to obtain a two-dimensional point cloud, wherein the two-dimensional coordinate system of the lidar is an XY coordinate system, and fitting the marker feature parameters in the two-dimensional coordinate system of the lidar using the least squares method, wherein the marker feature parameters include the slope along the distribution direction of the two-dimensional point cloud in the two-dimensional coordinate system of the lidar point cloud and the distance from the origin of the vehicle coordinate system in the normal direction.

[0102] like Figure 7 As shown, Figure 7 This diagram illustrates the positional relationship between the markers and the vehicle. Two LiDARs 2, located on either side of the front of vehicle 1, are represented by solid black circles. A dashed box represents the region of interest 3, within which two markers 4 are detected. The line l containing the markers in the LiDAR's two-dimensional coordinate system can be fitted using the least squares method. This line can then be used to determine the slope along the distribution direction of the two-dimensional point cloud in the LiDAR coordinate system, and the distance D from the origin of the vehicle coordinate system along the normal direction of the two-dimensional point cloud distribution direction can also be determined.

[0103] The slope of line l allows us to obtain the angle α between the line containing the sign and the vehicle's direction of travel. Angle α determines the vehicle's direction of travel. Even when the autonomous vehicle's direction of travel aligns with the sign's direction, angle α can be used to determine if the vehicle is veerging. This mathematical expression also provides the distance D from the intersection of the vehicle's centerline and the line containing its front to the line containing the sign. Distance D allows us to correct any deviation in the Y-direction.

[0104] Before extracting the point cloud features of the marker, this invention, based on the characteristics of mechanical LiDAR scan lines, introduces point cloud clustering and fitting at the scale of a single point cloud scan line. First, a round of point cloud screening is performed by setting ROI regions, thus avoiding large-scale point cloud clustering operations and reducing computational costs. Then, based on the pre-screening and cluster feature matching of single point cloud scan lines, point clouds belonging to the marker to be detected at specific heights can be extracted, reducing reliance on semantic information of the point clouds. Furthermore, for low-resolution radars, this invention can find as many point clouds belonging to the marker as possible. The method of this invention has low requirements for the computing power of data processing equipment and can achieve stable and reliable marker detection on automotive-grade chips.

[0105] This invention also provides a lidar point cloud data processing system, such as... Figure 8 As shown, it includes:

[0106] The first point cloud filtering module 801 is used to acquire the point cloud on the target marker scanned by the lidar within the target detection area, and select the point cloud containing the target marker by setting the ROI area.

[0107] The second point cloud filtering module 802 is used to extract and match point cloud features at the scale of a single laser scanning line on each scanning line of the lidar to obtain candidate point clouds.

[0108] The third point cloud screening module 803 is used to extract and match point cloud features at the overall scale of the candidate point cloud to obtain denoised and refined candidate point clouds.

[0109] The fourth point cloud filtering module 804 is used to perform RANSAC operation on the denoised and refined candidate point clouds to filter out the falsely detected point clouds and obtain the marker point clouds.

[0110] Optionally, the first point cloud filtering module 801 is specifically used to: obtain the lateral range of ROI by the lateral distance between the road boundary marker and the vehicle's autonomous driving route, wherein the lateral range of ROI is the range in the Y direction of the LiDAR three-dimensional coordinate system; obtain the longitudinal range of ROI by the detection range of the marker by the LiDAR, wherein the longitudinal range of ROI is the range in the X direction of the LiDAR three-dimensional coordinate system; obtain the height range of ROI by the range of change of the LiDAR scanning pitch angle, wherein the height range of ROI is the range in the Z direction of the LiDAR three-dimensional coordinate system; and select the point cloud containing the marker to be detected by the lateral range, longitudinal range, and height range of ROI.

[0111] Optionally, the second point cloud filtering module 802 specifically includes:

[0112] The scan line range determination unit is used to determine the scan line range corresponding to the target object based on the characteristics of the scan line obtained by the lidar and in combination with the installation position and pose information of the lidar.

[0113] The candidate point cloud cluster generation unit is used to filter point clouds within the scan line range and perform Euclidean clustering on the point clouds on each scan line after filtering to obtain candidate point cloud clusters.

[0114] The candidate point cloud generation unit is used to extract the features of candidate point cloud clusters and match them with the features of markers to obtain candidate point clouds.

[0115] Optional, the scan line range determination unit is specifically used for:

[0116] Calculate the angular range between the lidar and the target object scanned in the direction perpendicular to the lidar scanning direction, where the angular range is [θ]. min ,θ max ];

[0117] θ is determined according to the following formula. min :

[0118]

[0119] θ is determined according to the following formula. max :

[0120]

[0121] In the formula, θ min h is the angle between the line connecting the lidar and the lowest point of the target object and the ground. minh is the minimum distance between the marker to be inspected and the ground. L θ is the height of the lidar above the ground. max h is the angle between the line connecting the lidar and the highest point of the target object and the ground. max Let d be the maximum distance between the target object and the ground, and d be the distance between the lidar and the target object, where d ∈ [d]. min ,d max ], d min d represents the closest distance between the lidar and the target object within the lidar's detection range. max This represents the furthest distance between the lidar and the target object within the lidar's detection range.

[0122] The scan line range corresponding to the marker to be inspected is determined based on the included angle range.

[0123] Optionally, the candidate point cloud generation unit is specifically used to: extract the features of candidate point cloud clusters and match them with the features of the marker; and merge multiple candidate point cloud clusters that match the same marker on a single scan line using an interpolation method to obtain a candidate point cloud.

[0124] The third point cloud filtering module 803 specifically includes:

[0125] The first matching unit is used to select candidate point clouds whose point cloud feature values ​​are within a first preset range from the candidate point cloud by utilizing the structural features of the markers and / or the positional features of adjacent markers, so as to obtain the denoised and refined candidate point cloud. The structural features of the markers include at least one of the marker length, marker height, and marker curvature, and the positional features of adjacent markers include the spacing between adjacent markers.

[0126] The third point cloud filtering module 803 also includes:

[0127] The second matching unit is used to select candidate point clouds whose distance values ​​are within a second preset range from the candidate point clouds by using the distance between the centroid of the candidate point cloud and the vehicle, so as to obtain the denoised and refined candidate point cloud.

[0128] This invention also provides a lidar point cloud data processing system, such as... Figure 9 As shown, the system includes:

[0129] The first point cloud filtering module 901 is used to acquire the point cloud on the target marker scanned by the lidar within the target detection area, and select the point cloud containing the target marker by setting the ROI area.

[0130] Among them, the first point cloud filtering module 901 and Figure 8 The first point cloud filtering module 801 is similar, so it will not be described in detail here.

[0131] The second point cloud filtering module 902 is used to extract and match point cloud features at the scale of a single laser scanning line on each scanning line of the lidar to obtain candidate point clouds.

[0132] Among them, the second point cloud filtering module 902 and Figure 8 The second point cloud filtering module 802 is similar, so it will not be described in detail here.

[0133] The third point cloud screening module 903 is used to extract and match point cloud features at the overall scale of the candidate point cloud to obtain denoised and refined candidate point clouds.

[0134] Among them, the third point cloud filtering module 903 and Figure 8 The third point cloud filtering module 803 is similar, so it will not be described in detail here.

[0135] The fourth point cloud filtering module 904 is used to perform RANSAC operation on the denoised and refined candidate point clouds to filter out the falsely detected point clouds and obtain the marker point clouds.

[0136] Among them, the fourth point cloud filtering module 904 and Figure 8 The fourth point cloud filtering module 804 is similar, so it will not be described in detail here.

[0137] The marker feature parameter calculation module 905 is used to calculate the marker feature parameters using the marker point cloud, wherein the marker point cloud is a three-dimensional point cloud.

[0138] Optionally, the marker feature parameter calculation module is specifically used to: project the marker point cloud onto the two-dimensional coordinate system of the lidar to obtain a two-dimensional point cloud, wherein the two-dimensional coordinate system of the lidar is an XY coordinate system; and fit the marker feature parameters in the two-dimensional coordinate system of the lidar using the least squares method, wherein the marker feature parameters include the slope along the distribution direction of the two-dimensional point cloud in the two-dimensional coordinate system of the lidar point cloud and the distance from the origin of the vehicle coordinate system in the normal direction.

[0139] This invention provides a computer-readable storage medium storing a program thereon, which, when executed by a processor, implements the above-described lidar point cloud data processing method.

[0140] This invention provides an electronic device, such as... Figure 10 As shown, the electronic device 100 includes at least one processor 1001, at least one memory 1002 connected to the processor 1001, and a bus 1003; wherein the processor 1001 and the memory 1002 communicate with each other through the bus 1003; the processor 1001 is used to call program instructions in the memory 1002 to execute the above-described lidar point cloud data processing method. The electronic device in this document can be a server, PC, PAD, mobile phone, etc.

[0141] This application also provides a computer program product that, when executed on a data processing device, is adapted to perform a program that initializes the steps included in the above-described lidar point cloud data processing method.

[0142] This application is described with reference to flowchart illustrations and / or block diagrams of methods, systems, and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0143] In a typical configuration, the device includes one or more processors (CPUs), memory, and a bus. The device may also include input / output interfaces, network interfaces, etc.

[0144] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, like read-only memory (ROM) or flash RAM, and memory includes at least one memory chip. Memory is an example of computer-readable media.

[0145] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0146] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0147] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes the element.

[0148] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0149] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A method for processing lidar point cloud data, characterized in that, include: Acquire the point cloud of the target marker scanned by the lidar within the target detection area, and select the point cloud containing the target marker by setting the ROI region; The angle range between the laser radar and the target object scanned in a direction perpendicular to the laser radar scanning direction is calculated. The angle range is calculated based on the height of the laser radar above the ground, the height range of the target object above the ground, and the distance between the laser radar and the target object. The scan line range corresponding to the target object to be inspected is determined based on the included angle range; The point cloud is filtered within the scan line range, and Euclidean clustering is performed on the point cloud on each scan line after filtering to obtain candidate point cloud clusters. The features of the candidate point cloud clusters are extracted and matched with the features of the markers to obtain candidate point clouds. The candidate point cloud is subjected to point cloud feature extraction and matching at the overall point cloud scale to obtain a denoised and refined candidate point cloud. The RANSAC operation is performed on the denoised and refined candidate point cloud to filter out the falsely detected point clouds and obtain the marker point cloud.

2. The lidar point cloud data processing method according to claim 1, characterized in that, Also includes: Using the point cloud of the marker, the feature parameters of the marker are calculated, wherein the point cloud of the marker is a three-dimensional point cloud.

3. The lidar point cloud data processing method according to claim 2, characterized in that, The step of calculating the feature parameters of the markers using the marker point cloud includes: The point cloud of the markers is projected onto the two-dimensional coordinate system of the lidar to obtain a two-dimensional point cloud. The two-dimensional coordinate system of the lidar is the XY coordinate system. The feature parameters of the marker are obtained by least squares fitting in the two-dimensional coordinate system of the lidar. The feature parameters of the marker include the slope along the distribution direction of the two-dimensional point cloud in the two-dimensional coordinate system of the lidar point cloud and the distance from the origin of the vehicle coordinate system in the normal direction.

4. The lidar point cloud data processing method according to claim 3, characterized in that, The step of selecting a point cloud containing the marker to be inspected by setting a Region of Interest (ROI) includes: The lateral range of the Area of ​​Interest (ROI) is obtained by measuring the lateral distance between road boundary markers and the vehicle's autonomous driving route. The lateral range of the ROI is defined as the range along the Y-axis of the LiDAR's three-dimensional coordinate system. The longitudinal range of the Region of Interest (ROI) is obtained by measuring the detection range of the marker using the lidar, where the longitudinal range of the ROI is the range along the X direction of the lidar's three-dimensional coordinate system. The ROI height range is obtained by scanning the pitch angle range with a lidar scanner. The ROI height range is the range along the Z-axis of the lidar's three-dimensional coordinate system. The point cloud containing the marker to be inspected is selected by the ROI horizontal range, the ROI vertical range, and the ROI height range.

5. The lidar point cloud data processing method according to claim 1, characterized in that, The included angle range is [θ] min ,θ max ]; θ is determined according to the following formula. min : θ is determined according to the following formula. max : In the formula, θ min h is the angle between the line connecting the lowest point of the lidar and the target object and the ground. min h is the minimum distance between the marker to be inspected and the ground. L θ is the height of the lidar above the ground. max h is the angle between the line connecting the highest point of the lidar and the target object and the ground. max The distance between the target object and the ground is the maximum value, where d is the distance between the lidar and the target object, and d∈[d]. min ,d max ], d min d represents the closest distance between the lidar and the target object within the lidar's detection range. max The furthest distance between the lidar and the target object within the lidar's detection range; The scan line range corresponding to the marker to be inspected is determined based on the included angle range.

6. The lidar point cloud data processing method according to claim 1, characterized in that, The step of extracting features from the candidate point cloud clusters and matching them with marker features to obtain candidate point clouds includes: Features of the candidate point cloud clusters are extracted and matched with the features of the marker. Multiple candidate point cloud clusters that match the same marker on a single scan line are merged by interpolation to obtain the candidate point cloud.

7. The lidar point cloud data processing method according to claim 1, characterized in that, The process of extracting and matching point cloud features at the overall point cloud scale to obtain denoised and refined candidate point clouds includes: By utilizing the structural features of the markers and / or the positional features of adjacent markers, candidate point clouds whose point cloud feature values ​​are within a first preset range are selected from the candidate point clouds to obtain the denoised and refined candidate point clouds. The structural features of the markers include at least one of the marker length, marker height, and marker curvature, and the positional features of adjacent markers include the spacing between adjacent markers.

8. The lidar point cloud data processing method according to claim 1, characterized in that, The process of extracting and matching point cloud features at the overall point cloud scale to obtain denoised and refined candidate point clouds includes: Using the distance between the centroid of the candidate point cloud and the vehicle, candidate point clouds with distance values ​​within a second preset range are selected from the candidate point clouds to obtain the denoised and refined candidate point clouds.

9. A lidar point cloud data processing system, characterized in that, include: The first point cloud filtering module is used to acquire the point cloud on the target marker scanned by the lidar within the target detection area, and select the point cloud containing the target marker by setting the ROI area. The second point cloud filtering module is used to extract and match point cloud features at the scale of a single laser scan line for each scan line of the lidar to obtain candidate point clouds. The third point cloud screening module is used to extract and match point cloud features at the overall scale of the candidate point cloud to obtain denoised and refined candidate point clouds. The fourth point cloud filtering module is used to perform RANSAC operation on the denoised and refined candidate point cloud to filter out the falsely detected point cloud and obtain the marker point cloud. The second point cloud filtering module specifically includes: The scan line range determination unit is used to calculate the angle range between the laser radar and the target object in a direction perpendicular to the laser radar scanning direction. The angle range is calculated based on the height of the laser radar from the ground, the height range of the target object from the ground, and the distance between the laser radar and the target object. Based on the angle range, the scan line range corresponding to the target object is determined. The candidate point cloud cluster generation unit is used to filter the point cloud within the scan line range and perform Euclidean clustering on the point cloud on each scan line after filtering to obtain candidate point cloud clusters. The candidate point cloud generation unit is used to extract the features of candidate point cloud clusters and match them with the features of markers to obtain candidate point clouds.

Images