LiDAR-based method for detecting curbs and drivable areas on closed roads

Abstract

This invention provides a method for detecting road edges and drivable areas on closed roads based on lidar, comprising: acquiring lidar detection point cloud data; detecting point cloud data in groups based on voxel grids; voxel grids are obtained by dividing a predetermined detection space; a grid formed by projecting the combined voxel grids onto a ground plane as a raster image, and binarizing and marking the raster image according to the height distribution of the detected point cloud data within the voxel grids; and determining the road edge and drivable area based on the binarized and marked raster image. This invention improves the computational efficiency of road edge and drivable area detection by detecting point cloud data in groups using voxel grids, allowing point cloud data within each voxel grid to be processed in parallel. Simultaneously, binarizing and marking the detected point cloud data based on the height distribution within the voxel grids, and determining the road edge and drivable area using the binarized and marked raster image, further simplifies the computational steps and reduces the computational resource requirements.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving technology, and in particular to a method for detecting the curb and drivable area of ​​a closed road based on lidar. Background Technology

[0002] Road detection has long been considered a key technology in the field of autonomous driving, attracting widespread attention from researchers. However, autonomous driving decisions based solely on road detection may still be insufficient to handle some emergency situations. In fact, when driving a car, human drivers understand a situation by classifying obstacles and non-obstacles, not just recognizing the road. For autonomous driving, detecting these "flat areas" rather than road areas can provide a more comprehensive understanding of the decision-making process, allowing autonomous vehicles to behave more like human drivers.

[0003] Currently, the mainstream drivable area detection method is based on semantic segmentation of drivable areas using 2D RGB images. The detection results of 3D drivable areas need to be obtained by using the results of image depth estimation tasks, adding the camera calibration parameters to convert 2D pixels into 3D point clouds, and then performing post-processing.

[0004] 3D drivable regions are more practically significant for autonomous driving planning and control than 2D drivable regions. Although existing methods for 2D semantic segmentation of drivable regions are mature and accurate, they rely on depth estimation and camera calibration parameters to transform to a real-world 3D coordinate system. The accumulated errors in this process result in inaccurate 3D drivable regions. Furthermore, obtaining a 360-degree drivable region may require post-fusion of multiple camera results, which is complex and time-consuming.

[0005] To address these issues, existing technologies have proposed methods that use features on the scanning line of a mechanically rotating lidar point cloud for screening and clustering to obtain curb detection results.

[0006] However, with the widespread adoption of low-cost solid-state LiDAR, the efficiency of existing curb detection algorithms based on mechanically rotating LiDAR faces significant challenges. Solid-state LiDAR generates denser point clouds than traditional mechanically rotating LiDAR and lacks the characteristic of circular scan lines, causing existing algorithms based on rotating scan lines to fail. Furthermore, performing clustering algorithms on dense point clouds requires high computational resources and is slow.

[0007] Therefore, it is highly necessary and practical to provide a faster and more efficient method for detecting curbs and drivable areas that is adapted to dense point clouds of solid-state lidar. Summary of the Invention

[0008] This invention provides a method for detecting the curb and drivable area of ​​a closed road based on lidar, which solves the shortcomings of existing technologies that require high computing resources and have slow computing speed for dense point cloud clustering, and achieves efficient and accurate curb and drivable area detection.

[0009] This invention provides a method for detecting the curb and drivable area of ​​a closed road based on lidar, comprising:

[0010] Acquire point cloud data detected by lidar;

[0011] The detection point cloud data is grouped based on voxel grids; the voxel grids are obtained by dividing a predetermined detection space.

[0012] The grid formed by projecting the voxel squares onto the ground plane is combined into a grid map, and the grid map is binarized and labeled according to the height distribution of the detected point cloud data within the voxel squares.

[0013] Based on the grid map with binarized labels, the curb and drivable area are determined.

[0014] According to the present invention, a method for detecting the curb and drivable area of ​​a closed road based on lidar includes the step of determining the curb and drivable area based on the grid image with binarized markings, comprising:

[0015] Determine the curb endpoints based on the endpoint regions defined in the grid diagram;

[0016] Starting from the endpoint of the curb, move the curb sliding window established in the grid diagram, and determine the curb based on the binarized marker value of the grid within the curb sliding window;

[0017] The drivable area is determined based on the roadside.

[0018] According to the present invention, a method for detecting the curb and drivable area of ​​a closed road based on lidar includes the step of determining the curb endpoints based on the endpoint regions defined in the grid image, comprising:

[0019] A two-dimensional Cartesian coordinate system is established on the grid map with the current vehicle as the origin and the current vehicle's forward direction as the y-axis. An endpoint function is established for a specific part of the grid map. The value of the endpoint function is the sum of the binarized marker values ​​of the grid cells with the same x-coordinate.

[0020] Based on the endpoint function, the first position and the second position are determined as roadside endpoints;

[0021] The coordinates of the first position are (x1, 0), where x1 is the minimum value in the set of x-coordinates corresponding to the extreme values ​​of the endpoint function in the positive x-axis.

[0022] The coordinates of the second position are (x2, 0), where x2 is the maximum value in the set of x-coordinates corresponding to the extreme values ​​of the endpoint function in the negative half of the x-axis.

[0023] According to the present invention, a method for detecting the curb and drivable area of ​​a closed road based on lidar includes the following steps: moving a curb sliding window established in the grid map from the curb endpoint, and determining the curb based on the binarized marker values ​​of the grid within the curb sliding window:

[0024] A rectangular curb window is created in the grid diagram, and the position of the curb window is set to be the same as the x-coordinate of the center point of the curb window and the x-coordinate of the end point of the curb.

[0025] The position of the curb window is updated according to the binarized tag value of each grid in the curb window, and the center point of the curb window after the update is added to the curb point set;

[0026] The roadside extension direction is determined based on the binarized marker values ​​of each grid cell in the updated roadside window, and the roadside window is moved according to the roadside extension direction.

[0027] Return to the steps of updating the position of the curb window based on the binarized marker value of each grid in the curb window, and adding the center point of the updated curb window to the curb point set, until the set stop condition is met;

[0028] The curb is determined by fitting a set of curb points.

[0029] According to the present invention, a method for detecting the curb and drivable area of ​​a closed road based on lidar is provided, wherein the step of determining the drivable area based on the curb includes:

[0030] The grid map is divided into three parts using the two road edges as boundaries, and the part where the current vehicle is located is recorded as the road area;

[0031] The portion of the road area that is free of obstacles is defined as the drivable area.

[0032] The obstacle area is the portion of the first area excluding the second area; the first area is formed by the obstacle tangent and the curb in the grid diagram; the second area is formed by the obstacle tangent and the obstacle outline in the grid diagram.

[0033] The obstacle tangent is the tangent of the obstacle curve that passes through the current location of the vehicle; the obstacle curve is one or more curves formed by the combination of obstacle projection points; the obstacle projection points are the portion of the projection points of the detection point cloud data on the ground plane that are located within the road area.

[0034] The obstacle contour is the portion of the obstacle curve that is closest to the current vehicle's location, which is one of the two parts formed by dividing the obstacle curve by two obstacle tangent points; the obstacle tangent point is the point where the obstacle tangent line is tangent to the obstacle curve.

[0035] According to the present invention, a method for detecting the curb and drivable area of ​​a closed road based on lidar includes the step of binarizing and labeling the raster image according to the height distribution of the detected point cloud data within the voxel grid.

[0036] Calculate the height variance and three-dimensional surface curvature of the detected point cloud data within the voxel grid;

[0037] If the height variance of the detected point cloud data within the voxel grid is greater than a set variance threshold, and the three-dimensional surface curvature of the detected point cloud data within the voxel grid is greater than a set curvature threshold, then the grid formed by projecting the voxel grid onto the ground plane is marked as 1.

[0038] If the height variance of the detected point cloud data within the voxel grid is not greater than a set variance threshold, or the three-dimensional surface curvature of the detected point cloud data within the voxel grid is not greater than a set curvature threshold, then the grid formed by projecting the voxel grid onto the ground plane is marked as 0.

[0039] According to the present invention, a method for detecting the curb and drivable area of ​​a closed road based on lidar is provided, wherein the three-dimensional surface curvature σ satisfies:

[0040]

[0041] In the formula, λ0, λ1, and λ2 are the three eigenvectors of the covariance matrix C, and λ0 < λ1 < λ2; the covariance matrix C satisfies:

[0042]

[0043] In the formula, k is the number of detected point cloud data within the voxel grid; pi is the coordinate of the i-th detected point cloud data within the voxel grid. The mean value of the coordinates of the detected point cloud data within the voxel grid.

[0044] According to the present invention, a method for detecting the curb and drivable area of ​​a closed road based on lidar includes the following steps for acquiring lidar-detected point cloud data:

[0045] Acquire raw point cloud data from LiDAR;

[0046] From the original point cloud data, a single frame of point cloud data is selected to exclude the ground point cloud and form a single frame of detection point cloud data;

[0047] The detection point cloud data of a single frame is formed by superimposing a set number of adjacent frames.

[0048] According to the present invention, a method for detecting the curb and drivable area of ​​a closed road based on lidar includes the step of selecting a single frame of point cloud data from the original point cloud data to exclude ground point cloud data to form a single frame of detection point cloud data, which includes:

[0049] The surface plane is defined; the surface plane is the verification zone with the highest judgment weight; the verification zone is a verification area with a set thickness formed by translating the verification plane along the normal direction; the verification plane is obtained by fitting a set number of point clouds randomly sampled from a single frame of point cloud data; the judgment weight is the ratio of the number of point clouds within the verification zone to the number of point clouds outside the verification zone.

[0050] Point cloud data located within the ground plane are excluded from the single-frame point cloud data to form single-frame detection point cloud data.

[0051] According to the present invention, a method for detecting the curb and drivable area of ​​a closed road based on lidar is provided, wherein the step of determining the curb includes:

[0052] The first and second trail curves are corrected based on the average curvature of the first and second trail curves.

[0053] The curb is determined by the modified first and second curb curves.

[0054] The present invention also provides a lidar-based system for detecting curbs and drivable areas on closed roads, comprising:

[0055] The data acquisition module is used to acquire point cloud data detected by the lidar.

[0056] A voxel grouping module is used to group the detection point cloud data based on voxel grids; the voxel grids are obtained by dividing a predetermined detection space.

[0057] The binarization module is used to combine the voxel grids projected onto the ground plane to form a raster image, and to binarize and label the raster image according to the height distribution of the detected point cloud data within the voxel grids.

[0058] The region determination module is used to determine the curb and drivable area based on the grid map with binarized markings.

[0059] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the lidar-based method for detecting the curb and drivable area of ​​a closed road as described above.

[0060] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the lidar-based method for detecting curbs and drivable areas of closed roads as described above.

[0061] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the lidar-based method for detecting curbs and drivable areas of closed roads as described above.

[0062] The present invention provides a method for detecting road edges and drivable areas of closed roads based on lidar. By detecting point cloud data in voxel grid groups, the point cloud data in each voxel grid can be processed in parallel, improving the computational efficiency of road edge and drivable area detection. At the same time, the method performs binarization marking based on the height distribution of the detected point cloud data in the voxel grid, and determines the road edge and drivable area through the binarized grid map, further simplifying the calculation steps and reducing the demand for computing resources. Attached Figure Description

[0063] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0064] Figure 1 This is a flowchart illustrating the method for detecting the curb and drivable area of ​​a closed road based on lidar provided by the present invention.

[0065] Figure 2 This is a schematic diagram of the first binarized raster provided in an embodiment of the present invention;

[0066] Figure 3 This is a schematic diagram of the second binarized raster provided in an embodiment of the present invention;

[0067] Figure 4 This is a schematic diagram of signals used to determine the first position and the second position according to an embodiment of the present invention;

[0068] Figure 5 This is a schematic diagram of a curb sliding window provided in an embodiment of the present invention;

[0069] Figure 6 This is a schematic diagram of the first path edge after fitting, provided in an embodiment of the present invention;

[0070] Figure 7This is a schematic diagram of the fitted second path edge provided in an embodiment of the present invention;

[0071] Figure 8 This is a schematic diagram of the drivable area provided in an embodiment of the present invention;

[0072] Figure 9 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0073] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0074] The following is combined Figures 1-8 This invention describes a lidar-based method for detecting curbs and drivable areas on closed roads.

[0075] like Figure 1 As shown, this embodiment of the invention provides a method for detecting the curb and drivable area of ​​a closed road based on lidar, including:

[0076] Step 101: Obtain point cloud data detected by lidar;

[0077] Step 103: Group the detection point cloud data based on voxel grids; the voxel grids are obtained by dividing a predetermined detection space;

[0078] Step 105: Combine the voxel grids projected onto the ground plane to form a raster map, and binarize and label the raster map according to the height distribution of the detected point cloud data within the voxel grids.

[0079] Step 107: Determine the curb and drivable area based on the grid map with binarized markings.

[0080] The execution subject of this embodiment can be a computer program or a computer system.

[0081] In this embodiment, the voxel grid is set up as follows:

[0082] First, define the detection space as a cube with a predetermined length and width as its base and a predetermined height. Considering that in practical autonomous driving applications, the focus is on vehicles and common obstacles in the vertical direction, a reference value of 2 meters is set for the detection space's height. Considering the data collection characteristics of the current vehicle (and the LiDAR installed on it), one of the length or width sides of the detection space should pass through the current vehicle's location. The specific settings for the length and width depend on the LiDAR parameters and road conditions.

[0083] Subsequently, the detection space is segmented. The subsequent steps in this embodiment rely on the binarized labeling of the point cloud data within the voxel grid, and a raster map is formed using the projection of the voxel grid onto the ground plane. Therefore, the height of the voxel grid can be simply set to a cube with the same height as the detection space to avoid overlapping projections of the voxel grids onto the ground plane. Furthermore, considering the base dimensions of the voxel grid, and taking into account both computational accuracy (the "resolution" of the binarized raster) and computational efficiency, this embodiment sets both dimensions to 0.3 meters.

[0084] It should be noted that the above-mentioned voxel grid setting should not be construed as a limitation of this embodiment; this setting is merely a preferred embodiment.

[0085] In some practical application scenarios, the number of voxel squares formed by the above scheme is 128*512. That is, in the forward direction of the current vehicle, point cloud data within a range of 512*0.3=153.6m is considered, and in the left and right lateral directions of the current vehicle, point cloud data within a range of 128*0.3=38.4m is considered.

[0086] Based on the above voxel grid configuration, the grid diagram in this embodiment consists of 128*512 square grids.

[0087] The binarization label of the raster is determined by the height distribution of the point cloud data within the corresponding voxel square. The more discrete the height distribution of the point cloud, the higher the probability that the voxel square and the corresponding raster may be components of the roadside. Therefore, a threshold for the degree of dispersion of the height distribution is set by an empirical formula. Rasters above the threshold are set to 1, and rasters below the threshold are set to 0. This yields a raster map with a binarized distribution. When determining the roadside location based on the raster map, the consideration of rasters set to 0 can be weakened.

[0088] It is worth noting that the 0 and 1 in the binarization markers should not be interpreted as a limitation on this embodiment. The purpose of the markers is to divide the grid into two categories based on the discreteness of the height distribution. Therefore, grids above the threshold can be set to 0, and grids below the threshold can be set to 1. Alternatively, other numbers, letters, colors, or shadows can be used instead of 0 and 1. This embodiment does not limit this.

[0089] Once the curb is determined, the space inside the curb can be understood as the road area. Then, based on the point cloud data within the road area, other vehicles or obstacles within the road can be analyzed to obtain a conclusion about the drivable area.

[0090] The beneficial effects of this embodiment are as follows:

[0091] The lidar-based method for detecting road edges and drivable areas on closed roads provided in this embodiment detects point cloud data in voxel grid groups, enabling subsequent calculations to be performed in parallel within each voxel grid, thus improving the computational efficiency of road edge and drivable area detection. Simultaneously, it performs binarization marking based on the height distribution of the detected point cloud data within the voxel grid, and determines the road edge and drivable area using the binarized grid map, further simplifying the computational steps and reducing the computational resource requirements.

[0092] According to the above embodiments, in this embodiment:

[0093] The step of determining the curb and drivable area based on the grid map with binarized labels includes:

[0094] Determine the curb endpoints based on the endpoint regions defined in the grid diagram;

[0095] Starting from the endpoint of the curb, move the curb sliding window established in the grid diagram, and determine the curb based on the binarized marker value of the grid within the curb sliding window;

[0096] The drivable area is determined based on the roadside.

[0097] In this embodiment, the curb sliding window is used to traverse and move from the starting point of the curb to better fit the curb curve.

[0098] The step of determining the curb endpoints based on the endpoint regions defined in the grid diagram includes:

[0099] A two-dimensional Cartesian coordinate system is established on the grid map with the current vehicle as the origin and the current vehicle's forward direction as the y-axis. An endpoint function is established for a specific part of the grid map. The value of the endpoint function is the sum of the binarized marker values ​​of the grid cells with the same x-coordinate.

[0100] Based on the endpoint function, the first position and the second position are determined as roadside endpoints;

[0101] The coordinates of the first position are (x1, 0), where x1 is the minimum value in the set of x-coordinates corresponding to the extreme values ​​of the endpoint function in the positive x-axis.

[0102] The coordinates of the second position are (x2, 0), where x2 is the maximum value in the set of x-coordinates corresponding to the extreme values ​​of the endpoint function in the negative half of the x-axis.

[0103] In this embodiment, the Cartesian coordinate system is established for ease of explanation. In actual use, the setting of the origin and coordinate axes can be replaced, and the transformed coordinates can reach the same conclusion as the coordinates in this embodiment through only simple mathematical changes.

[0104] In this embodiment, the actual physical meaning of the curb endpoints can be understood as the two curb starting points on the same y-coordinate that are closer to the current vehicle in the grid diagram; (in certain specific scenarios) it can also be understood as the two curb endpoints that are farther away from the current vehicle in the grid diagram; or it can be understood as including both two curb starting points and two curb endpoints.

[0105] Correspondingly, in the subsequent curb sliding window traversal steps of this embodiment, the curb sliding window can move from the curb starting point, or from the curb ending point, or from both the curb starting point and the curb ending point respectively.

[0106] In this embodiment, the step of moving the curb sliding window established in the grid map from the curb endpoint and determining the curb based on the binarized marker value of the grid within the curb sliding window includes:

[0107] A rectangular curb window is created in the grid diagram, and the position of the curb window is set to be the same as the x-coordinate of the center point of the curb window and the x-coordinate of the end point of the curb.

[0108] The position of the curb window is updated according to the binarized tag value of each grid in the curb window, and the center point of the curb window after the update is added to the curb point set;

[0109] The roadside extension direction is determined based on the binarized marker values ​​of each grid cell in the updated roadside window, and the roadside window is moved according to the roadside extension direction.

[0110] Return to the steps of updating the position of the curb window based on the binarized marker value of each grid in the curb window, and adding the center point of the updated curb window to the curb point set, until the set stop condition is met;

[0111] The curb is determined by fitting a set of curb points.

[0112] In this embodiment, the size of the curb sliding window should be larger than the size of the grid. In this embodiment, a rectangular sliding window that can accommodate 3*4=12 grids is selected.

[0113] In the step of updating the position of the curb sliding window based on the binarized marker values ​​of each grid in the curb sliding window, the update method can be:

[0114] Calculate the centroid of the set of 1-cell grids (i.e., grids with a highly discrete distribution above a threshold) in the curb window, and update the position of the curb window so that its center point coincides with the centroid.

[0115] In the step of determining the roadside extension direction based on the binarized marker values ​​of each grid cell in the updated roadside sliding window, the method for determining the roadside extension direction can be as follows:

[0116] The roadside extension line is obtained by fitting a grid with 1 grid in the center of the roadside sliding window (i.e., a grid with a highly discrete distribution above the threshold). Based on the current vehicle orientation, the starting position of the roadside sliding window, and any combination of the starting and ending positions of the roadside, the roadside extension direction is determined (the roadside extension direction is collinear with the roadside extension line).

[0117] In this embodiment, the step of determining the drivable area based on the curb includes:

[0118] The grid map is divided into three parts using the two road edges as boundaries, and the part where the current vehicle is located is recorded as the road area;

[0119] The portion of the road area that is free of obstacles is defined as the drivable area.

[0120] The obstacle area is the portion of the first area excluding the second area; the first area is formed by the obstacle tangent and the curb in the grid diagram; the second area is formed by the obstacle tangent and the obstacle outline in the grid diagram.

[0121] The obstacle tangent is the tangent of the obstacle curve that passes through the current location of the vehicle; the obstacle curve is one or more curves formed by the combination of obstacle projection points; the obstacle projection points are the portion of the projection points of the detection point cloud data on the ground plane that are located within the road area.

[0122] The obstacle contour is the portion of the obstacle curve that is closest to the current vehicle's location, which is one of the two parts formed by dividing the obstacle curve by two obstacle tangent points; the obstacle tangent point is the point where the obstacle tangent line is tangent to the obstacle curve.

[0123] The steps for determining the drivable area in this embodiment can be simply understood as follows:

[0124] First, define the road area by using the curb;

[0125] Second, identify obstacles within the road area based on the three-dimensional point cloud data (the obstacles may be other vehicles).

[0126] Third, based on the Raycasting algorithm, the area where obstacles block the road is identified;

[0127] Fourth, within the road area, areas obstructed by road obstacles are excluded to obtain the final drivable area.

[0128] In this embodiment, the step of determining the curb includes:

[0129] The first and second trail curves are corrected based on the average curvature of the first and second trail curves.

[0130] The curb is determined by the modified first and second curb curves.

[0131] The beneficial effects of this embodiment are as follows:

[0132] This embodiment can directly obtain accurate roadside and drivable areas from a top-down view, and is suitable for both mechanical rotating and solid-state LiDAR. Because it is based on voxelization processing, it utilizes various geometric features of point clouds within voxels and fits road edge curves using a windowing method, making it more efficient and accurate than existing methods. Then, raycasting is used to remove areas obscured by objects in the top-down view, accurately obtaining the true drivable area.

[0133] According to any of the above embodiments, in this embodiment:

[0134] The step of binarizing and labeling the raster image based on the height distribution of the detected point cloud data within the voxel grid includes:

[0135] Calculate the height variance and three-dimensional surface curvature of the detected point cloud data within the voxel grid;

[0136] If the height variance of the detected point cloud data within the voxel grid is greater than a set variance threshold, and the three-dimensional surface curvature of the detected point cloud data within the voxel grid is greater than a set curvature threshold, then the grid formed by projecting the voxel grid onto the ground plane is marked as 1.

[0137] If the height variance of the detected point cloud data within the voxel grid is not greater than a set variance threshold, or the three-dimensional surface curvature of the detected point cloud data within the voxel grid is not greater than a set curvature threshold, then the grid formed by projecting the voxel grid onto the ground plane is marked as 0.

[0138] The three-dimensional surface curvature σ satisfies:

[0139]

[0140] In the formula, λ0, λ1, and λ2 are the three eigenvectors of the covariance matrix C, and λ0 < λ1 < λ2; the covariance matrix C satisfies:

[0141]

[0142] In the formula, k is the number of detected point cloud data within the voxel grid; p i The coordinates of the i-th detection point cloud data within the voxel grid; The mean value of the coordinates of the detected point cloud data within the voxel grid.

[0143] The beneficial effects of this embodiment are as follows:

[0144] This embodiment provides a method for quantifying the distribution of three-dimensional point clouds within a voxel grid, and adds verification based on the curvature of three-dimensional surfaces on the basis of height distribution to make the calculation results more accurate.

[0145] According to any of the above embodiments, in this embodiment:

[0146] The steps for acquiring point cloud data detected by lidar include:

[0147] Acquire raw point cloud data from LiDAR;

[0148] From the original point cloud data, a single frame of point cloud data is selected to exclude the ground point cloud and form a single frame of detection point cloud data;

[0149] The detection point cloud data of a single frame is formed by superimposing a set number of adjacent frames.

[0150] The step of selecting a single frame of point cloud data from the original point cloud data to exclude ground point cloud data and forming a single frame of detection point cloud data includes:

[0151] The surface plane is defined; the surface plane is the verification zone with the highest judgment weight; the verification zone is a verification area with a set thickness formed by translating the verification plane along the normal direction; the verification plane is obtained by fitting a set number of point clouds randomly sampled from a single frame of point cloud data; the judgment weight is the ratio of the number of point clouds within the verification zone to the number of point clouds outside the verification zone.

[0152] Point cloud data located within the ground plane are excluded from the single-frame point cloud data to form single-frame detection point cloud data.

[0153] The beneficial effects of this embodiment are as follows:

[0154] This embodiment fits the ground plane equation based on the RANSAC method and performs ground removal processing on the point cloud, which reduces the interference of the ground point cloud on subsequent calculations and effectively improves the computational efficiency of drivable area detection.

[0155] Meanwhile, this embodiment, based on the voxel grid segmentation method, reduces the demand for computing resources, thus supporting the computation of denser effective point clouds (i.e., detection point cloud data excluding ground point clouds). Therefore, by superimposing a set number of adjacent frames of the single-frame detection point cloud data, detection point cloud data is formed for subsequent calculations, resulting in more accurate detection results for road edges and drivable areas.

[0156] The following is a complete embodiment with an overall explanation based on actual calculations. This embodiment uses a LiDAR sensor to directly detect the drivable area in the 3D coordinate system of a BEV. The input to this embodiment is the point cloud of the current frame and the target detection bounding box, which can be obtained from manual annotation or a target detection model.

[0157] This embodiment specifically includes the following steps:

[0158] 1. Point Cloud Preprocessing: First, the RANSAC method is used to fit the ground plane equation to remove the ground from the point cloud. Then, points within the target detection boxes are removed using the output of the target detector or manually annotated detection boxes. The remaining points are used as the preprocessed point cloud for the current frame. At this point, map positioning information can be used to overlay the preprocessed point clouds from the previous 1-5 frames onto the current frame to make the point cloud denser and the edge features more obvious.

[0159] 2. Road boundary line detection:

[0160] 2.1 Perform a columnar voxelization operation on the point cloud processed in the first step, for example, a 0.3m x 0.3m grid, to obtain a 3D point cloud mapped to a 2D BEV map, and initialize the drivable area to 1.

[0161] 2.2 For points within the same voxel, calculate the height variance and 3D surface curvature in the vertical ground coordinates as geometric features of the voxel. Then, apply a threshold to each voxel and perform an AND operation to obtain voxels that may have road boundaries, in the form of a binary map under BEV (Boundary Elevation Vehicle).

[0162] Formula for calculating the curvature of a 3D surface:

[0163]

[0164] Lambda(λ0, λ1, λ2) are the eigenvectors of the covariance matrix C.

[0165] The curvature of a 3D surface corresponds to the weight of the minimum eigenvector, with the value varying between 0 and 1. The lower the value, the flatter the surface. This feature is generally more robust and stable than the orthogonal features of the surface.

[0166] Height variance

[0167] Height variance represents the smoothness of a surface; the larger the value, the less smooth the surface.

[0168]

[0169] Figure 2 , Figure 3 Two different scenarios are provided, with the binarized raster images shown. The black parts are raster cells set to 1, and the white parts are raster cells set to 0.

[0170] 2.3 Plot the signal diagram by summing the lower half of the binary graph downwards, and find the first peak that appears on the left and right sides of the vehicle as the starting point of the road edge.

[0171] Figure 4 The diagram shows a signal used to determine the first and second positions in a specific application scenario.

[0172] 2.4 Starting from the initial point, slide windows upwards on the binary graph, updating the center point of each window to the centroid of the non-zero voxel of the current window. Finally, use the center point of each window to fit the road boundary curve to obtain the left and right road edge curves. Since the edges of highway scenes are generally parallel to each other, the curvature can be averaged to maintain the consistency of the curvature of the left and right edge curves.

[0173] Figure 5 This is a schematic diagram of a curb sliding window, where the black rectangular blocks represent the curb sliding window.

[0174] Figure 6 , Figure 7 Two different scenarios are provided, each with a fitted roadside diagram.

[0175] 2.5 Set the pixels outside the road boundary in the BEV map to 0 to represent non-drivable areas.

[0176] 3. Raycasting: On the BEV map, raycasting is performed on the corner points of the target box starting from the center point of the vehicle. The area where the line of sight is obscured by the target box is set to 0, and the remaining area with a value of 1 is the drivable area.

[0177] A diagram of the drivable area after raycasting is shown below. Figure 8 As shown.

[0178] The beneficial effects of this embodiment are as follows:

[0179] This embodiment can directly obtain accurate road edges and drivable areas from a top-down view, and is suitable for both mechanical rotating and solid-state LiDAR. Because it is based on voxelization processing, it utilizes various geometric features of point clouds within voxels and fits road edge curves using a windowing method, making it more efficient and accurate than existing methods. Then, raycasting is used to remove areas obscured by objects in the top-down view, accurately obtaining the true drivable area. The resulting road edges and drivable areas can be used as ground truth for neural network learning or output to the downstream planning and control module of autonomous driving perception.

[0180] The following describes the lidar-based detection device for closed road curbs and drivable areas provided by the present invention. The lidar-based detection device for closed road curbs and drivable areas described below can be referred to in correspondence with the lidar-based detection method for closed road curbs and drivable areas described above.

[0181] This invention provides a lidar-based system for detecting curbs and drivable areas on closed roads, comprising:

[0182] The data acquisition module is used to acquire point cloud data detected by the lidar.

[0183] A voxel grouping module is used to group the detection point cloud data based on voxel grids; the voxel grids are obtained by dividing a predetermined detection space.

[0184] The binarization module is used to combine the voxel grids projected onto the ground plane to form a raster image, and to binarize and label the raster image according to the height distribution of the detected point cloud data within the voxel grids.

[0185] The region determination module is used to determine the curb and drivable area based on the grid map with binarized markings.

[0186] The data acquisition module includes:

[0187] The raw data submodule is used to acquire raw point cloud data from the LiDAR.

[0188] The ground point cloud exclusion submodule is used to select a single frame of point cloud data from the original point cloud data to exclude the ground point cloud and form a single frame of detection point cloud data.

[0189] The single-frame overlay submodule is used to overlay a set number of adjacent frames of the single-frame detection point cloud data to form detection point cloud data.

[0190] The ground point cloud exclusion submodule includes:

[0191] A surface plane determination unit is used to determine the surface plane; the surface plane is the verification zone with the highest judgment weight; the verification zone is a verification area with a set thickness formed by translating the verification plane along the normal direction; the verification plane is obtained by fitting a set number of point clouds randomly sampled from a single frame of point cloud data; the judgment weight is the ratio of the number of point clouds within the verification zone to the number of point clouds outside the verification zone.

[0192] The point cloud exclusion unit is used to exclude point cloud data located in the ground plane from the single frame point cloud data to form a single frame detection point cloud data.

[0193] The binarization module includes:

[0194] The variance curvature calculation submodule is used to calculate the height variance and three-dimensional surface curvature of the detected point cloud data within the voxel grid;

[0195] The binarization judgment submodule is used for:

[0196] If the height variance of the detected point cloud data within the voxel grid is greater than a set variance threshold, and the three-dimensional surface curvature of the detected point cloud data within the voxel grid is greater than a set curvature threshold, then the grid formed by projecting the voxel grid onto the ground plane is marked as 1.

[0197] If the height variance of the detected point cloud data within the voxel grid is not greater than a set variance threshold, or the three-dimensional surface curvature of the detected point cloud data within the voxel grid is not greater than a set curvature threshold, then the grid formed by projecting the voxel grid onto the ground plane is marked as 0.

[0198] The three-dimensional surface curvature σ satisfies:

[0199]

[0200] In the formula, λ0, λ1, and λ2 are the three eigenvectors of the covariance matrix C, and λ0 < λ1 < λ2; the covariance matrix C satisfies:

[0201]

[0202] In the formula, k is the number of detected point cloud data within the voxel grid; p i The coordinates of the i-th detection point cloud data within the voxel grid; The mean value of the coordinates of the detected point cloud data within the voxel grid.

[0203] The region determination module includes:

[0204] The curb endpoint submodule is used to determine the curb endpoint based on the endpoint region set in the grid diagram;

[0205] The curb determination submodule is used to move a curb sliding window established in the grid map from the curb endpoint and determine the curb based on the binarized marker value of the grid in the curb sliding window.

[0206] The drivable area determination submodule is used to determine the drivable area based on the roadside.

[0207] The curb endpoint submodule includes:

[0208] The endpoint function unit is used to establish a two-dimensional Cartesian coordinate system on the grid map with the current vehicle as the origin and the current vehicle's forward direction as the y-axis, and to establish endpoint functions for a specified portion of the grid map; the value of the endpoint function is the sum of the binarized marker values ​​of the grids with the same x-coordinate;

[0209] An endpoint determination unit is used to determine the first position and the second position as roadside endpoints based on the endpoint function.

[0210] The coordinates of the first position are (x1, 0), where x1 is the minimum value in the set of x-coordinates corresponding to the extreme values ​​of the endpoint function in the positive x-axis.

[0211] The coordinates of the second position are (x2, 0), where x2 is the maximum value in the set of x-coordinates corresponding to the extreme values ​​of the endpoint function in the negative half of the x-axis.

[0212] The curb determination submodule includes:

[0213] The curb sliding window initial position unit is used to establish a rectangular curb sliding window in the grid diagram, and set the position of the curb sliding window to be the same as the x-coordinate of the center point of the curb sliding window and the x-coordinate of the end point of the curb.

[0214] The curb sliding window position update unit is used to update the position of the curb sliding window according to the binarized marker value of each grid in the curb sliding window, and add the center point of the curb sliding window after the updated position to the curb point set;

[0215] A curb sliding window moving unit is used to determine the curb extension direction based on the binarized marker values ​​of each grid in the curb sliding window after the position is updated, and to move the curb sliding window according to the curb extension direction.

[0216] The sliding window iteration unit is used to return to the steps of updating the position of the curb sliding window according to the binarized marker value of each grid in the curb sliding window, and adding the center point of the curb sliding window after the updated position to the curb point set, until the set stop condition is met;

[0217] The curb fitting and determination unit is used to fit and determine the curb based on the set of curb points.

[0218] The drivable area determination submodule includes:

[0219] The road area unit is used to divide the grid map into three parts with the two road edges as boundaries, and the part where the current vehicle is located is recorded as the road area;

[0220] An obstacle removal zone unit is used to determine a portion of the road area where the obstacle removal zone is located as a drivable area.

[0221] The obstacle area is the portion of the first area excluding the second area; the first area is formed by the obstacle tangent and the curb in the grid diagram; the second area is formed by the obstacle tangent and the obstacle outline in the grid diagram.

[0222] The obstacle tangent is the tangent of the obstacle curve that passes through the current location of the vehicle; the obstacle curve is one or more curves formed by the combination of obstacle projection points; the obstacle projection points are the portion of the projection points of the detection point cloud data on the ground plane that are located within the road area.

[0223] The obstacle contour is the portion of the obstacle curve that is closest to the current vehicle's location, which is one of the two parts formed by dividing the obstacle curve by two obstacle tangent points; the obstacle tangent point is the point where the obstacle tangent line is tangent to the obstacle curve.

[0224] Figure 9 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 9 As shown, the electronic device may include a processor 910, a communication interface 920, a memory 930, and a communication bus 940. The processor 910, communication interface 920, and memory 930 communicate with each other via the communication bus 940. The processor 910 can call logical instructions in the memory 930 to execute a method for detecting curbs and drivable areas on closed roads based on lidar. This method includes: acquiring lidar detection point cloud data; grouping the detection point cloud data based on voxel grids; the voxel grids are obtained by dividing a predetermined detection space; combining the voxel grids into a grid formed by projecting them onto a ground plane to form a raster image, and binarizing the raster image according to the height distribution of the detection point cloud data within the voxel grids; and determining the curb and drivable area based on the binarized raster image.

[0225] Furthermore, the logical instructions in the aforementioned memory 830 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0226] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the lidar-based method for detecting road edges and drivable areas on closed roads provided by the above methods. The method includes: acquiring lidar detection point cloud data; grouping the detection point cloud data based on voxel grids; the voxel grids are obtained by dividing a predetermined detection space; combining the voxel grids into a grid formed by projecting them onto a ground plane to form a grid image, and binarizing the grid image according to the height distribution of the detection point cloud data within the voxel grids; and determining the road edge and drivable area based on the binarized grid image.

[0227] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the lidar-based method for detecting curbs and drivable areas of closed roads, as described above. This method includes: acquiring lidar detection point cloud data; grouping the detection point cloud data based on voxel grids; wherein the voxel grids are obtained by segmenting a predetermined detection space; combining the voxel grids into a grid formed by projecting them onto a ground plane to form a raster image, and binarizing the raster image according to the height distribution of the detection point cloud data within the voxel grids; and determining the curb and drivable area based on the binarized raster image.

[0228] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0229] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0230] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting the curb and drivable area of ​​a closed road based on lidar, characterized in that, include: Acquire point cloud data detected by lidar; The detection point cloud data is based on voxel grid grouping; The voxel grid is obtained by dividing a predetermined detection space; The grid formed by projecting the voxel squares onto the ground plane is combined into a grid map, and the grid map is binarized and labeled according to the height distribution of the detected point cloud data within the voxel squares. Based on the grid map with binarized labels, determine the curb and drivable area; The drivable area is determined based on the curb, and the steps for determining the drivable area based on the curb include: The grid map is divided into three parts using the two road edges as boundaries, and the part where the current vehicle is located is recorded as the road area; The portion of the road area that is free of obstacles is defined as the drivable area. The obstacle area is the portion of the first area excluding the second area; the first area is formed by the obstacle tangent and the curb in the grid diagram; the second area is formed by the obstacle tangent and the obstacle outline in the grid diagram. The obstacle tangent is the tangent of the obstacle curve that passes through the current location of the vehicle; the obstacle curve is one or more curves formed by the combination of obstacle projection points; the obstacle projection points are the portion of the projection points of the detection point cloud data on the ground plane that are located within the road area. The obstacle contour is the portion of the obstacle curve that is closest to the current vehicle's location, which is one of the two parts formed by dividing the obstacle curve by two obstacle tangent points; the obstacle tangent point is the point where the obstacle tangent line is tangent to the obstacle curve.

2. The method for detecting curbs and drivable areas of closed roads based on lidar according to claim 1, characterized in that, The step of determining the curb and drivable area based on the grid map with binarized labels includes: Determine the curb endpoints based on the endpoint regions defined in the grid diagram; Starting from the endpoint of the curb, move the curb sliding window established in the grid diagram, and determine the curb based on the binarized marker value of the grid within the curb sliding window; The drivable area is determined based on the roadside.

3. The method for detecting curbs and drivable areas of closed roads based on lidar according to claim 2, characterized in that, The step of determining the curb endpoints based on the endpoint regions defined in the grid diagram includes: A two-dimensional Cartesian coordinate system is established on the grid map with the current vehicle as the origin and the current vehicle's forward direction as the y-axis. An endpoint function is established for a specific part of the grid map. The value of the endpoint function is the sum of the binarized marker values ​​of the grid cells with the same x-coordinate. Based on the endpoint function, the first position and the second position are determined as roadside endpoints; The coordinates of the first position are (x1, 0), where x1 is the minimum value in the set of x-coordinates corresponding to the extreme values ​​of the endpoint function in the positive x-axis. The coordinates of the second position are (x2, 0), where x2 is the maximum value in the set of x-coordinates corresponding to the extreme values ​​of the endpoint function in the negative half of the x-axis.

4. The method for detecting curbs and drivable areas of closed roads based on lidar according to claim 3, characterized in that, The step of moving the curb sliding window established in the grid map from the curb endpoint and determining the curb based on the binarized marker value of the grid within the curb sliding window includes: A rectangular curb window is created in the grid diagram, and the position of the curb window is set to be the same as the x-coordinate of the center point of the curb window and the x-coordinate of the end point of the curb. The position of the curb window is updated according to the binarized tag value of each grid in the curb window, and the center point of the curb window after the update is added to the curb point set; The roadside extension direction is determined based on the binarized marker values ​​of each grid cell in the updated roadside window, and the roadside window is moved according to the roadside extension direction. Return to the steps of updating the position of the curb window based on the binarized marker value of each grid in the curb window, and adding the center point of the updated curb window to the curb point set, until the set stop condition is met; The curb is determined by fitting a set of curb points.

5. The method for detecting curbs and drivable areas of closed roads based on lidar according to claim 1, characterized in that, The step of binarizing and labeling the raster image based on the height distribution of the detected point cloud data within the voxel grid includes: Calculate the height variance and three-dimensional surface curvature of the detected point cloud data within the voxel grid; If the height variance of the detected point cloud data within the voxel grid is greater than a set variance threshold, and the three-dimensional surface curvature of the detected point cloud data within the voxel grid is greater than a set curvature threshold, then the grid formed by projecting the voxel grid onto the ground plane is marked as 1. If the height variance of the detected point cloud data within the voxel grid is not greater than a set variance threshold, or the three-dimensional surface curvature of the detected point cloud data within the voxel grid is not greater than a set curvature threshold, then the grid formed by projecting the voxel grid onto the ground plane is marked as 0.

6. The method for detecting curbs and drivable areas of closed roads based on lidar according to claim 5, characterized in that, The three-dimensional surface curvature σ satisfies: In the formula, λ0, λ1, and λ2 are the three eigenvectors of the covariance matrix C, and λ0 < λ1 < λ2; the covariance matrix C satisfies: In the formula, k is the number of detected point cloud data within the voxel grid; p i The coordinates of the i-th detection point cloud data within the voxel grid; The mean value of the coordinates of the detected point cloud data within the voxel grid.

7. The method for detecting curbs and drivable areas of closed roads based on lidar according to any one of claims 1 to 6, characterized in that, The steps for acquiring point cloud data detected by lidar include: Acquire raw point cloud data from LiDAR; From the original point cloud data, a single frame of point cloud data is selected to exclude the ground point cloud and form a single frame of detection point cloud data; The detection point cloud data of a single frame is formed by superimposing a set number of adjacent frames.

8. The method for detecting curbs and drivable areas of closed roads based on lidar according to claim 7, characterized in that, The step of selecting a single frame of point cloud data from the original point cloud data to exclude ground point cloud data and forming a single frame of detection point cloud data includes: The surface plane is defined; the surface plane is the verification zone with the highest judgment weight; the verification zone is a verification area with a set thickness formed by translating the verification plane along the normal direction; the verification plane is obtained by fitting a set number of point clouds randomly sampled from a single frame of point cloud data; the judgment weight is the ratio of the number of point clouds within the verification zone to the number of point clouds outside the verification zone. Point cloud data located within the ground plane are excluded from the single-frame point cloud data to form single-frame detection point cloud data.

9. The method for detecting curbs and drivable areas of closed roads based on lidar according to any one of claims 1 to 6, characterized in that, The step of determining the curb includes: The first and second trail curves are corrected based on the average curvature of the first and second trail curves. The curb is determined by the modified first and second curb curves.

10. A system for detecting curbs and drivable areas on closed roads based on lidar, characterized in that, include: The data acquisition module is used to acquire point cloud data detected by the lidar. Voxel grouping module, used to group the detected point cloud data based on voxel grids; The voxel grid is obtained by dividing a predetermined detection space; The binarization module is used to combine the voxel grids projected onto the ground plane to form a raster image, and to binarize and label the raster image according to the height distribution of the detected point cloud data within the voxel grids. The region determination module is used to determine the roadside and drivable area based on the grid map with binarized markings; The area determination module includes a drivable area determination submodule, which includes: The road area unit is used to divide the grid map into three parts with the two road edges as boundaries, and the part where the current vehicle is located is recorded as the road area; An obstacle removal zone unit is used to determine a portion of the road area where the obstacle removal zone is located as a drivable area. The obstacle area is the portion of the first area excluding the second area; the first area is formed by the obstacle tangent and the curb in the grid diagram; the second area is formed by the obstacle tangent and the obstacle outline in the grid diagram. The obstacle tangent is the tangent of the obstacle curve that passes through the current location of the vehicle; the obstacle curve is one or more curves formed by the combination of obstacle projection points; the obstacle projection points are the portion of the projection points of the detection point cloud data on the ground plane that are located within the road area. The obstacle contour is the portion of the obstacle curve that is closest to the current vehicle's location, which is one of the two parts formed by dividing the obstacle curve by two obstacle tangent points; the obstacle tangent point is the point where the obstacle tangent line is tangent to the obstacle curve.

11. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the lidar-based method for detecting curbs and drivable areas on closed roads as described in any one of claims 1 to 9.

12. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the lidar-based method for detecting curbs and drivable areas of closed roads as described in any one of claims 1 to 9.

13. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the lidar-based method for detecting curbs and drivable areas of closed roads as described in any one of claims 1 to 9.

Images