An intelligent driving drivable area detection method and device

Abstract

The application provides a smart driving drivable area detection method and device, the method comprising: acquiring point cloud data of multiple laser radars under the same timestamp; performing fusion processing on multiple frames of point cloud data to obtain a fused point cloud set; determining non-ground point cloud data from the fused point cloud set according to the z-direction coordinate values of the fused point cloud data contained in the fused point cloud set, so as to obtain a non-ground point cloud set composed of the determined non-ground point cloud data; mapping the non-ground point cloud set to a pre-established target polar coordinate system, so as to determine the closest distances between obstacles around the vehicle under full angles and the vehicle respectively according to the non-ground point cloud set under the target polar coordinate system; and determining the drivable area of the vehicle according to the closest distances between the obstacles around the vehicle under full angles and the vehicle respectively. The application uses multiple laser radars to collect point cloud data, greatly improves the detection distance, and has better environmental adaptability. Since it does not need the assistance of high-precision maps, structured roads and other conditions, the autonomous detection performance is high.

Description

Technical Field

[0001] This application relates to the field of intelligent driving technology, and in particular to a method and apparatus for detecting drivable areas for intelligent driving. Background Technology

[0002] Intelligent driving drivable area detection is a technology that provides safety boundaries for intelligent driving by detecting the area where the vehicle can drive through the surrounding environment.

[0003] Existing methods for detecting drivable areas in intelligent driving mainly include two types: one is to use cameras to detect road types and identify roads in order to determine drivable areas; the other is to use LiDAR and high-precision maps to obtain boundary points in order to solve for drivable areas.

[0004] However, methods for road type target detection using cameras have limited detection distance, high light requirements, are generally suitable for structured scenes, and have poor environmental adaptability; while methods for drivable area detection using lidar and high-precision maps require regular maintenance and updates of the high-precision maps. Summary of the Invention

[0005] In view of this, this application provides a method and apparatus for detecting drivable areas in intelligent driving, which solves the problem of poor environmental adaptability in the method of road type target detection based on cameras, and the problem of needing to regularly maintain and update high-precision maps in the method of drivable area detection based on lidar and high-precision maps. The technical solution is as follows:

[0006] A method for detecting drivable areas in intelligent driving includes:

[0007] Point cloud data at the same timestamp from multiple lidar sensors is acquired. The point cloud data is collected by lidar sensors installed on the vehicle, and all point cloud data is transformed to the vehicle coordinate system. The point cloud data includes multiple frames of point cloud data.

[0008] Multi-frame point cloud data are fused to obtain a fused point cloud set;

[0009] Based on the z-coordinate values ​​of the fused point cloud data contained in the fused point cloud set, non-ground point cloud data is determined from the fused point cloud set to obtain a non-ground point cloud set composed of the determined non-ground point cloud data;

[0010] The non-ground point cloud is mapped to the pre-established target polar coordinate system. Based on the non-ground point cloud in the target polar coordinate system, the closest distance between the vehicle and the obstacles around the vehicle at all angles is determined. The target polar coordinate system is a target polar coordinate system established with the origin of the vehicle coordinate system as the pole.

[0011] The drivable area of ​​the vehicle is determined by the closest distance between the vehicle and obstacles at all angles around it.

[0012] Optionally, the multi-frame point cloud data is fused to obtain a fused point cloud set, including:

[0013] Multiple frames of point cloud data are stitched together into a single frame of point cloud data according to a preset lidar sequence to obtain a fused point cloud set;

[0014] or,

[0015] For each point cloud data contained in a multi-frame point cloud data set, calculate the corresponding horizontal and vertical angles of that point cloud data, and fill the corresponding point cloud data into the corresponding point cloud raster based on the corresponding horizontal and vertical angles; thus filling each point cloud data contained in the multi-frame point cloud data into the corresponding point cloud raster.

[0016] For each point cloud raster, if the point cloud raster contains multiple point cloud data, the centroid of the multiple point cloud data is taken as the fused point cloud data under the point cloud raster; so as to obtain a fused point cloud set composed of the fused point cloud data under all point cloud rasteres.

[0017] Optionally, based on the z-coordinate values ​​of the fused point cloud data contained in the fused point cloud set, non-ground point cloud data is determined from the fused point cloud set, including:

[0018] Remove the fused point cloud data containing the abnormal z-coordinate values ​​in the fused point cloud set, and use the fused point cloud set after removing the abnormal data as the target fused point cloud set;

[0019] Based on the z-coordinate values ​​of the fused point cloud data contained in the target fused point cloud set, ground point cloud data is determined from the target fused point cloud set to obtain a ground point cloud set to be updated composed of the determined ground point cloud data;

[0020] Determine the fitted plane equation based on the ground point cloud set to be updated;

[0021] Calculate the distance between each fused point cloud data in the target fused point cloud set and the fitted plane equation. Determine the fused point cloud data whose calculated distance is less than a preset first distance threshold as ground point cloud data. Use the determined ground point cloud data as the updated ground point cloud set.

[0022] Increment the update iteration count by 1, and determine whether the updated iteration count after incrementing by 1 has reached the preset total number of iterations. If not, use the updated ground point cloud set as the ground point cloud data to be updated, and return to execute the determination of the fitting plane equation based on the ground point cloud set to be updated, until the update iteration count reaches the total number of iterations. Then, use the other point cloud data in the target fused point cloud set other than the updated ground point cloud set as non-ground point cloud data. The initial update iteration count is 0.

[0023] Optionally, ground point cloud data is determined from the target fused point cloud set based on the z-coordinate values ​​of the fused point cloud data contained in the target fused point cloud set, including:

[0024] Based on the z-coordinate values ​​of the fused point cloud data contained in the target fused point cloud set, a preset number of fused point cloud data are selected from the target fused point cloud set, and the average z-coordinate value of the preset number of fused point cloud data is calculated.

[0025] For each fused point cloud data in the target fused point cloud set, if the difference between the z-coordinate value and the z-average value of the fused point cloud data is less than a preset difference threshold, then the fused point cloud data is determined to be ground point cloud data.

[0026] Optionally, the fitted plane equation is determined based on the ground point cloud set to be updated, including:

[0027] Calculate the x-axis average, y-axis average, and z-axis average of the ground point cloud to be updated;

[0028] Based on the x-axis average, y-axis average, and z-axis average of the ground point cloud to be updated, determine the covariance matrix and solve for several eigenvalues ​​and eigenvectors of the covariance matrix.

[0029] The smallest eigenvalue is determined from a number of eigenvalues, and the eigenvector corresponding to the smallest eigenvalue is used as the normal vector of the fitting plane.

[0030] Based on the ground point cloud set to be updated and the normal vector of the fitted plane, the equation of the fitted plane is solved.

[0031] Optionally, the target polar coordinate system includes multiple angular grids, and each angular grid includes multiple radial grids;

[0032] The non-ground point cloud is mapped to a pre-established target polar coordinate system. Based on the non-ground point cloud in the target polar coordinate system, the nearest distances between obstacles around the vehicle and the vehicle at all angles are determined, including:

[0033] Based on the x-coordinate and y-coordinate values ​​of each non-ground point cloud data in the non-ground point cloud set, calculate the horizontal angle and two-dimensional projection distance corresponding to each non-ground point cloud data.

[0034] Based on the horizontal angle and two-dimensional projection distance corresponding to each non-ground point cloud data in the non-ground point cloud set, each non-ground point cloud data is mapped to the target polar coordinate system to obtain the non-ground point cloud set in the target polar coordinate system.

[0035] For each angle grid contained in the target polar coordinate system, determine the non-empty radial grid closest to the pole from that angle grid, calculate the average distance between the non-ground point cloud data contained in the non-empty radial grid and the pole, and use it as the closest distance between the obstacle and the vehicle at the corresponding angle of that angle grid; so as to obtain the closest distance between the obstacle and the vehicle at all angles around the vehicle.

[0036] Optionally, the drivable area of ​​the vehicle can be determined based on the closest distances to obstacles around the vehicle from all angles, including:

[0037] Based on the aggregation of non-ground points and the nearest distance, determine the probability that each grid in the preset grid map contains an obstacle;

[0038] The drivable area for a vehicle is determined based on the probability that each grid cell in the grid map contains an obstacle.

[0039] Optionally, based on the non-ground point cluster and the nearest distance, determine the probability that each grid cell in the preset grid map contains an obstacle, including:

[0040] Based on the x-coordinate and y-coordinate values ​​of each non-ground point cloud data in the non-ground point cloud set and the preset grid resolution, each non-ground point cloud data is mapped to the corresponding grid contained in the grid map. The grid resolution is used to divide the grid map into several grids.

[0041] Based on the target fitting plane equation and the preset second distance threshold, the non-ground point cloud data contained in the raster map is determined as low point data or high point data, wherein the target fitting plane equation is the fitting plane equation determined in the last iteration;

[0042] Based on the low point data and high point data contained in each grid cell in the grid map, as well as the low point data contained in adjacent grid cells, the preset grid occupancy probability and the target fitting plane equation, determine the mixed occupancy probability corresponding to each grid cell.

[0043] Based on the coordinate transformation relationship between the rear axle center of the vehicle and the origin of the grid map, the distance between each grid in the grid map and the rear axle center of the vehicle is calculated. Based on the calculated distance and the closest distance between the obstacle and the vehicle at the corresponding angle of each grid, the initial occupancy probability of each grid is determined.

[0044] Based on the mixed occupancy probability and initial occupancy probability of each grid cell in the grid map, determine the probability that each grid cell in the grid map contains an obstacle.

[0045] Optionally, based on the low-point and high-point data contained in each grid cell in the raster map, as well as the low-point data contained in adjacent grid cells, the preset grid occupancy probability, and the target fitting plane equation, the mixed occupancy probability corresponding to each grid cell is determined, including:

[0046] Calculate the number of high-point data points contained in each grid cell in the grid map, and determine the high-point occupancy probability corresponding to each grid cell in the grid map based on the number of high-point data points and the preset grid occupancy probability.

[0047] Based on the low point data contained in each grid cell in the grid map, the low point data contained in adjacent grid cells, and the target fitted plane equation, determine the low point occupancy probability corresponding to each grid cell in the grid map.

[0048] Based on the high point occupancy probability and low point occupancy probability of each grid in the grid map, determine the mixed occupancy probability of each grid.

[0049] A smart driving drivable area detection device, comprising:

[0050] The point cloud data acquisition module is used to acquire point cloud data at the same timestamp, wherein the point cloud data is collected by the LiDAR installed on the vehicle;

[0051] The non-ground point cloud set determination module is used to determine non-ground point cloud data from the point cloud data based on the z-coordinate values ​​of the fused point cloud data contained in the fused point cloud set, so as to obtain a non-ground point cloud set composed of the determined non-ground point cloud data.

[0052] The closest distance determination module is used to map the non-ground point cloud to a pre-established target polar coordinate system, so as to determine the closest distance between the vehicle and obstacles around the vehicle at all angles based on the non-ground point cloud in the target polar coordinate system. The target polar coordinate system is a polar coordinate system established with the origin of the vehicle coordinate system as the pole.

[0053] The drivable area determination module is used to determine the drivable area of ​​the vehicle based on the closest distance between the vehicle and obstacles around the vehicle from all angles.

[0054] As can be seen from the above technical solution, the intelligent driving drivable area detection method provided in this application first acquires point cloud data from multiple lidars at the same timestamp, then fuses the multi-frame point cloud data to obtain a fused point cloud set, then determines non-ground point cloud data from the fused point cloud set based on the z-coordinate values ​​of the fused point cloud data contained in the fused point cloud set, so as to obtain a non-ground point cloud set composed of the determined non-ground point cloud data, then maps the non-ground point cloud set to a pre-established target polar coordinate system, so as to determine the closest distance between the vehicle and obstacles around the vehicle at all angles based on the non-ground point cloud set in the target polar coordinate system, and finally determines the drivable area of ​​the vehicle based on the closest distance between the vehicle and obstacles around the vehicle at all angles. This application uses multiple lidars to collect point cloud data, which greatly improves the detection distance and enhances environmental adaptability. Furthermore, this application can determine the closest distance between obstacles around the vehicle and the vehicle at all angles. Based on the closest distance, the boundaries of the drivable area around the vehicle at each angle can be accurately obtained. The entire process relies solely on the point cloud data collected by lidars and does not require additional conditions such as high-precision maps or structured roads, resulting in high autonomous detection performance. Attached Figure Description

[0055] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0056] Figure 1 A flowchart illustrating a method for detecting drivable areas for intelligent driving, provided in an embodiment of this application;

[0057] Figure 2 A schematic diagram of a point cloud grid provided in an embodiment of this application;

[0058] Figure 3 A schematic diagram of the target polar coordinate system provided in the embodiments of this application;

[0059] Figure 4 This is a schematic diagram of the intelligent driving drivable area detection device provided in an embodiment of this application;

[0060] Figure 5 This is a hardware structure block diagram of the intelligent driving drivable area detection device provided in the embodiments of this application. Detailed Implementation

[0061] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0062] In view of the problems existing in the prior art, the inventors of this case have conducted in-depth research and finally proposed an intelligent driving drivable area detection method. Optionally, this intelligent driving drivable area detection method can be used in the field of autonomous driving navigation such as path planning and path control. The following embodiments will provide a detailed description of the intelligent driving drivable area detection method provided in this application.

[0063] Please see Figure 1 The diagram illustrates a flowchart of an intelligent driving drivable area detection method provided in an embodiment of this application. This intelligent driving drivable area detection method may include:

[0064] Step S101: Obtain point cloud data from multiple lidar sensors at the same timestamp.

[0065] The point cloud data is collected by the lidar installed on the vehicle, and all point cloud data is transformed into the vehicle coordinate system; the point cloud data includes multiple frames of point cloud data.

[0066] In this step, point cloud data can be collected using multiple lidars installed on the vehicle, such as collecting point cloud data based on the main lidar and the left and right side lidars.

[0067] After the lidar collects point cloud data, a timestamp can be determined for each lidar-collected point cloud data set via a host computer, thus recording the time when the lidar collected the point cloud data. It is understood that multiple lidars can collect several frames of point cloud data at multiple times. This application can match several frames of point cloud data collected by multiple lidars according to their corresponding timestamps (i.e., synchronize the time of several frames of point cloud data) to obtain multiple frames of point cloud data under each timestamp. The "point cloud data under the same timestamp" obtained in this step refers to multiple frames of point cloud data under any given timestamp.

[0068] It should be understood that different LiDARs are located in different positions. To facilitate subsequent processing of point cloud data collected by different LiDARs, multiple frames of point cloud data collected by multiple LiDARs can be spatially synchronized. This spatial synchronization refers to transforming the multiple frames of point cloud data collected by multiple LiDARs into a unified coordinate system, such as the vehicle coordinate system. Here, the vehicle coordinate system refers to a coordinate system with the center of the rear axle of the vehicle as the origin. In other words, the point cloud data obtained in this step at the same timestamp are all point cloud data transformed into the vehicle coordinate system.

[0069] Step S102: Perform fusion processing on the multi-frame point cloud data to obtain a fused point cloud set.

[0070] The aforementioned multi-frame point cloud data was obtained by multiple lidars capturing images of obstacles around the vehicle from multiple angles. To facilitate subsequent processing, this step can be used to fuse the multi-frame point cloud data into a single frame of point cloud data. The point cloud data contained in the fused single frame of point cloud data constitutes the fused point cloud set in this step.

[0071] Step S103: Based on the z-coordinate values ​​of the fused point cloud data contained in the fused point cloud set, determine the non-ground point cloud data from the fused point cloud set to obtain a non-ground point cloud set composed of the determined non-ground point cloud data.

[0072] Specifically, a frame of point cloud data acquired by a lidar may include at least one point cloud data point, which typically includes some ground point cloud data and some non-ground point cloud data. In this embodiment, the z-coordinate value of the fused point cloud data is positively correlated with the ground distance of the fused point cloud data. Therefore, this step can determine the non-ground point cloud data from the fused point cloud set obtained in the previous steps.

[0073] Step S104: Map the non-ground point cloud to the pre-established target polar coordinate system, so as to determine the closest distance between the vehicle and the obstacles around the vehicle at all angles based on the non-ground point cloud in the target polar coordinate system.

[0074] The target polar coordinate system is a polar coordinate system established with the origin of the vehicle coordinate system as the pole.

[0075] Understandably, due to the characteristics of LiDAR object detection, non-ground point cloud data is highly likely to represent obstacles. This step establishes a target polar coordinate system with the origin of the vehicle coordinate system (i.e., the center of the vehicle's rear axle) as the pole. Then, each non-ground point cloud data point is mapped to the target polar coordinate system. Based on the distance between the non-ground point cloud data and the pole of the target polar coordinate system, the closest distance between the non-ground point cloud data and the vehicle can be determined. In other words, the closest distance between obstacles and the vehicle at each angle around the vehicle can be determined. For example, if there are three obstacles (a, b, and c) at 90° of the vehicle, and obstacle b is closest to the vehicle, then the distance between obstacle b and the vehicle is determined as the closest distance between obstacles and the vehicle at 90°.

[0076] In this step, the nearest distance refers to the distance to the nearest obstacle from the center of the vehicle's rear axle in each angular direction. By calculating the nearest distance, this step can obtain the boundary information of the vehicle's drivable area, thus intuitively narrowing down the vehicle's driving range.

[0077] Step S105: Determine the drivable area of ​​the vehicle based on the closest distances between the vehicle and obstacles around it from all angles.

[0078] In this step, the nearest distance characterizes the boundary information of the vehicle's drivable area. For example, an autonomous vehicle can only travel to the corresponding nearest distance at a certain angle and cannot exceed that nearest distance. Therefore, after determining the nearest distances between the vehicle and obstacles at all angles around the vehicle, the vehicle's drivable area can be obtained.

[0079] The intelligent driving drivable area detection method provided in this application first acquires point cloud data from multiple LiDARs at the same timestamp. Then, it fuses multiple frames of point cloud data to obtain a fused point cloud set. Next, based on the z-coordinate values ​​of the fused point cloud data contained in the fused point cloud set, non-ground point cloud data is determined from the fused point cloud set to obtain a non-ground point cloud set composed of the determined non-ground point cloud data. This non-ground point cloud set is then mapped to a pre-established target polar coordinate system. Based on the non-ground point cloud set in the target polar coordinate system, the nearest distances between obstacles around the vehicle at all angles are determined. Finally, based on the nearest distances between obstacles around the vehicle at all angles, the drivable area of ​​the vehicle is determined. This application uses multiple LiDARs to collect point cloud data, significantly increasing the detection range and improving environmental adaptability. Furthermore, this application can determine the nearest distances between obstacles around the vehicle at all angles, accurately obtaining the boundaries of the drivable area around the vehicle at each angle based on these nearest distances. The entire process relies solely on the point cloud data collected by the LiDARs, requiring no additional conditions such as high-precision maps or structured roads, resulting in high autonomous detection performance.

[0080] An embodiment of this application describes the process of "step S102, fusing multiple frames of point cloud data to obtain a fused point cloud set".

[0081] In this embodiment, the fused point cloud set includes multiple fused point cloud data.

[0082] Optionally, this embodiment can use both GPU-based parallel multithreading and CPU-based single-threading methods for fusion processing.

[0083] In this embodiment, the process of "fusion processing of multi-frame point cloud data" can be implemented in a variety of ways. This application provides, but is not limited to, the following two implementation methods.

[0084] The first implementation method is to use an unstructured fusion method to fuse multi-frame point cloud data.

[0085] Here, the unstructured fusion process includes: stitching multiple frames of point cloud data into a single frame of point cloud data according to a preset lidar sequence to obtain a fused point cloud set.

[0086] This implementation method directly stitches together a frame of point cloud data collected by each lidar in a preset order. During stitching, point cloud data corresponding to the same obstacle collected by different lidars will not be merged into a single point cloud data. In other words, the total number of point clouds contained in the multi-frame point cloud data is the same as the total number of point clouds contained in the fused point cloud set.

[0087] For example, assuming there are N lidars, numbered 0 to N-1 according to a preset lidar order, the number of point cloud data points contained in a frame of point cloud data collected by the i-th lidar is n. i Where i∈[0,N-1], then the number of point clouds contained in the fused point cloud set is

[0088] The second implementation method is to use a structured fusion approach to fuse multi-frame point cloud data.

[0089] In this implementation, the structured fusion process includes the following steps a1 to a2:

[0090] Step a1: For each point cloud data contained in the multi-frame point cloud data, calculate the horizontal and vertical angles corresponding to the point cloud data, and fill the point cloud data into the corresponding point cloud grid according to the horizontal and vertical angles corresponding to the point cloud data, so that each point cloud data contained in the multi-frame point cloud data is filled into the corresponding point cloud grid.

[0091] Specifically, firstly, the three-dimensional space is divided into W columns in the horizontal direction (0–360°) and H rows in the vertical direction (0–180°), resulting in... Figure 2The point cloud grid is shown. Then, for each point cloud data contained in each frame of the multi-frame point cloud (a frame of point cloud data usually includes multiple point cloud data), the horizontal angle corresponding to the point cloud data is calculated using the x-coordinate and y-coordinate values ​​of the point cloud data. The vertical angle corresponding to the point cloud data is calculated using the x-coordinate, y-coordinate and z-coordinate values ​​of the point cloud data. Then, the point cloud data is filled into the corresponding point cloud grid according to the calculated horizontal and vertical angles.

[0092] Optionally, if the three-dimensional coordinates of a point cloud data are (x, y, z), it should be noted that in this application, the three-dimensional coordinates of the point cloud data (x, y, z) represent the three-dimensional coordinates returned when the laser beam of the lidar hits an object. The origin of this coordinate system is the center of the rear axle of the vehicle in this application. The X, Y, Z coordinate system in this application is a coordinate system with the center of the rear axle of the vehicle as the origin. x represents the distance from the object point to the lateral plane of the rear axle of the vehicle, y represents the distance from the object point to the longitudinal plane of the rear axle of the vehicle, and z represents the height of the object point from the center point of the rear axle. Then, the horizontal angle corresponding to the point cloud data can be calculated using the formula acrtan(y / x), and the vertical angle corresponding to the point cloud data can be calculated using the formula acrtan(z / sqrt(x^2+y^2)). Based on this, when the point cloud data is filled into the corresponding point cloud grid, the column number of the corresponding point cloud grid is acrtan(y / x) / 360*W, and the row number is obtained by querying the prior table based on the calculated vertical angle (each mechanical lidar will correspond to a prior table of the vertical angle of the laser beam, that is, the vertical angle of each beam is fixed at the factory).

[0093] Step a2: For each point cloud grid, if the point cloud grid contains multiple point cloud data, then take the centroid of the multiple point cloud data as the fused point cloud data under the point cloud grid; so as to obtain the fused point cloud set composed of the fused point cloud data under all point cloud grids.

[0094] Since the detection angles of multiple lidars may overlap (for example, the main lidar and the left lidar overlap at the upper left corner of the vehicle, and the main lidar and the right lidar overlap at the upper right corner of the vehicle), if an obstacle happens to be in the area of ​​overlapping angles, the same obstacle will be detected by multiple lidars. In this case, fusion processing is performed, and multiple point cloud data corresponding to the same obstacle may be filled into the same point cloud grid.

[0095] In this step, if all the point cloud data from the LiDAR are filled into the corresponding point cloud grid, and a point cloud grid contains multiple point cloud data, then the centroids of the multiple point cloud data (i.e., the average values ​​of the x, y, and z directions of the multiple point cloud data) are used as the fused point cloud data under that point cloud grid.

[0096] The fusion processing method of this implementation can merge multiple point cloud data into one point cloud data, so that the total number of point clouds contained in the multi-frame point cloud data is greater than the total number of point clouds contained in the fused point cloud set.

[0097] A comparison of the two implementation methods reveals that the first method has the advantages of low time and space complexity and good preservation of environmental information. However, it cannot obtain the spatial neighborhood relationship of each point in the point cloud frame and retain as many environmental features detected by each LiDAR as possible. Storing the fused point cloud also requires a large amount of memory. The second method retains the three-dimensional spatial neighborhood information of each point in the point cloud frame (this information can be used to determine whether any two fused point cloud data are adjacent and in which direction they are adjacent). However, it cannot achieve a good balance between spatial complexity and environmental realism and may lose some features detected by a certain LiDAR.

[0098] In summary, the first implementation method is more suitable for scenarios where point cloud data processed through drivable areas will be directly used by subsequent perception tasks; the second implementation method is more suitable for scenarios where point cloud data processed through drivable areas will not be directly used by subsequent perception tasks, and has better performance, avoiding unnecessary performance waste.

[0099] It should be noted that the two fusion processing methods mentioned above are merely examples. Other fusion methods may also be used, and this application does not limit them.

[0100] In one optional embodiment, the process of "step S103, determining non-ground point cloud data from the fused point cloud set based on the z-coordinate values ​​of the fused point cloud data contained in the fused point cloud set" is described.

[0101] This embodiment can divide the fused point cloud into non-ground point cloud data and ground point cloud data.

[0102] Optionally, the process of "determining non-ground point cloud data from the fused point cloud set based on the z-coordinate values ​​of the fused point cloud data contained in the fused point cloud set" in this embodiment includes the following steps b1 to b5:

[0103] Step b1: Remove the fusion point cloud data containing abnormal z-coordinate values ​​from the fusion point cloud set, and use the fusion point cloud set after removing abnormal data as the target fusion point cloud set.

[0104] Due to factors such as uneven ground (e.g., a raised center and lower sides), and the inability to guarantee 100% horizontal installation accuracy of radar and radar vehicles, environmental noise (spectral reflection noise) exists in the fused point cloud. The z-coordinate value of this environmental noise usually differs significantly from the z-coordinate value of the non-noise fused point cloud data. For example, if the z-coordinate values ​​of the fused point cloud data in the fused point cloud are -20, -1, 1, 2, and 3, then the fused point cloud data with a z-coordinate value of -20 is abnormal data and can be removed from the fused point cloud to obtain the target fused point cloud.

[0105] Step b2: Based on the z-coordinate values ​​of the fused point cloud data contained in the target fused point cloud set, determine the ground point cloud data from the target fused point cloud set to obtain the ground point cloud set to be updated composed of the determined ground point cloud data.

[0106] In this step, the z-coordinate value of a fused point cloud data is related to the distance of the fused point cloud data from the ground. Based on the z-coordinate value of each fused point cloud data contained in the target fused point cloud set, it can be determined whether each fused point cloud data is ground point cloud data or non-ground point cloud data, thereby obtaining the ground point cloud set to be updated.

[0107] Optionally, the process of "determining ground point cloud data from the target fused point cloud set based on the z-coordinate values ​​of the fused point cloud data contained in the target fused point cloud set" includes the following steps b21 to b22:

[0108] Step b21: Based on the z-coordinate values ​​of the fused point cloud data contained in the target fused point cloud set, select a preset number of fused point cloud data from the target fused point cloud set, and calculate the average z-coordinate of the preset number of fused point cloud data.

[0109] In this step, the z-axis average value of a preset number of fused point cloud data reflects, to some extent, the height difference between the ground and the center of the vehicle's rear axle.

[0110] Optionally, this step can sort the fused point cloud data in the target fused point cloud set according to the z-coordinate values ​​of the fused point cloud data from smallest to largest, then select a preset number of fused point cloud data with the smallest z-coordinate values, and calculate the average z-coordinate values ​​of the preset number of fused point cloud data to obtain the z-average value.

[0111] Step b22: For each fused point cloud data in the target fused point cloud set, if the difference between the z-coordinate value and the z-average value of the fused point cloud data is less than a preset difference threshold, then the fused point cloud data is determined to be ground point cloud data.

[0112] Optionally, in step b21, where the fused point cloud data in the target fused point cloud set is sorted in ascending order of z-coordinate values, this step can calculate the difference between the z-coordinate value and the z-average value of each fused point cloud data point in the sorting order, and then compare the calculated difference with a preset difference threshold (which can be used in this embodiment). This indicates the difference threshold (optionally, this difference threshold can be set via the ground clearance of the vehicle's rear axle center) for comparison, to select vehicles with a difference less than the difference threshold. The fused point cloud data was determined to be ground point cloud data.

[0113] Since this step can perform difference calculation and comparison according to the sorting order, if the difference between the z-coordinate value and the z-average value of a fused point cloud data is greater than the difference threshold... Therefore, the difference between the z-coordinate value and the z-average value of the subsequently fused point cloud data will inevitably be greater than the difference threshold. Therefore, when the difference between the calculated z-coordinate value and the z-average value of a fused point cloud data exceeds a difference threshold... When the calculation is completed, no further calculation is required, saving calculation time and improving calculation efficiency.

[0114] Step b3: Determine the fitted plane equation based on the ground point cloud set to be updated.

[0115] In this step, principal component analysis can be used to achieve plane fitting and solve the planar model (i.e., the fitted plane equation) of the ground point cloud to be updated. Here, the planar model refers to fitting the ground point cloud to be updated into a plane and expressing the fitted plane in the form of ax+by+cz+d=0. The principal component analysis method in this application is an existing algorithm and is a statistical method that transforms a set of potentially correlated variables into a set of linearly uncorrelated variables through orthogonal transformation. The transformed set of variables is called principal components.

[0116] Optionally, the process of "determining the fitted plane equation based on the ground point cloud set to be updated" may include steps b31 to b34:

[0117] Step b31: Calculate the x-axis average, y-axis average, and z-axis average of the ground point cloud to be updated.

[0118] Specifically, each ground point cloud data in the ground point cloud set to be updated is a ground point cloud data represented by a three-dimensional coordinate value (i.e., x-coordinate value, y-coordinate value, and z-coordinate value), namely (x, y, z). This step can calculate the average value of the x-coordinate value, y-coordinate value, and z-coordinate value of each ground point cloud data in the ground point cloud set to be updated, respectively, to obtain the x-coordinate average value, y-coordinate average value, and z-coordinate average value.

[0119] Step b32: Determine the covariance matrix based on the x-axis average, y-axis average, and z-axis average of the ground point cloud to be updated, and solve for several eigenvalues ​​and eigenvectors of the covariance matrix.

[0120] In this step, based on the x-axis average, y-axis average, and z-axis average of the ground point cloud to be updated, the covariance matrix (a 3x3 matrix) of the ground point cloud to be updated in the x, y, and z directions can be calculated, and several eigenvalues ​​and eigenvectors of the covariance matrix can be solved. Typically, three eigenvalues ​​and their corresponding three eigenvectors can be obtained.

[0121] Step b33: Determine the smallest eigenvalue from several eigenvalues, and use the eigenvector corresponding to the smallest eigenvalue as the normal vector of the fitting plane.

[0122] It is understandable that the smaller the eigenvalue, the smaller the discriminative power (discretion) of the point cloud in that dimension. Therefore, the eigenvector corresponding to the smallest eigenvalue can be used as the normal vector of the fitting plane.

[0123] Step b34: Solve for the equation of the fitted plane based on the ground point cloud set to be updated and the normal vector of the fitted plane.

[0124] Specifically, this step can be based on the centroids (i.e., the average values ​​in the x, y, and z directions) of all ground point cloud data contained in the ground point cloud set to be updated, combined with the normal vector of the fitting plane, to solve for the fitting plane equation of the form ax + by + cz + d = 0.

[0125] Step b4: Calculate the distance between each fused point cloud data in the target fused point cloud set and the fitted plane equation. Determine the fused point cloud data whose calculated distance is less than the preset first distance threshold as ground point cloud data, and use the determined ground point cloud data as the updated ground point cloud set.

[0126] Optionally, this step can determine a first distance threshold based on the ground point cloud segmentation requirements. For ease of subsequent description, the first distance threshold will be expressed as...

[0127] After determining the first distance threshold Then, if the distance between a fused point cloud data and the fitted plane equation is less than a first distance threshold... The fused point cloud data is likely to be ground point cloud data, from which the updated ground point cloud set can be obtained.

[0128] The method used in this step to calculate the distance between the fused point cloud data and the fitted plane equation is an existing technique and will not be described in detail here.

[0129] Step b5: Increment the update iteration count by 1, and determine whether the update iteration count after incrementing by 1 has reached the preset total number of iterations. If not, use the updated ground point cloud set as the ground point cloud data to be updated, and return to execute the determination of the fitting plane equation based on the ground point cloud set to be updated, until the update iteration count reaches the total number of iterations. Then, use the other point cloud data in the target fused point cloud set other than the updated ground point cloud set as non-ground point cloud data.

[0130] The initial number of update iterations is 0.

[0131] To ensure the accuracy of the determined fitting plane equation, this embodiment can perform several iterations, such as 2 to 3 iterations. Based on this, the initial number of iterations is set to 0. Each time the ground point cloud to be updated is updated, the iteration count is incremented by 1. It is then determined whether the incremented iteration count reaches the preset total number of iterations (since determining the fitting plane equation and b4 is relatively time-consuming, and due to real-time limitations, it is difficult to set the iteration count too high; therefore, this embodiment generally sets the total number of iterations to 2 or 3). If the incremented iteration count is less than or equal to the total number of iterations, another iteration is performed, using the updated ground point cloud as the ground point cloud data to be updated, and the fitting plane equation is re-determined based on the ground point cloud data to be updated.

[0132] When the number of update iterations after adding 1 is greater than the total number of iterations, it means that no further iterations are needed. At this time, the updated ground point cloud set determined by the last iteration can be taken as the final ground point cloud set, and the other point cloud data in the target fused point cloud set other than the final ground point cloud set can be determined as non-ground point cloud data.

[0133] It should be noted that the process of determining non-ground point cloud data shown in steps b1 to b5 above is only an example and is not intended to limit this application. For example, this application can also directly determine non-ground point cloud data based solely on steps b1 to b2.

[0134] This embodiment provides a method for determining non-ground point cloud data when point cloud data at the same timestamp includes multiple frames of point cloud data. Using multiple frames of point cloud data can minimize the vehicle's detection blind spot, making the determined non-ground point cloud data more accurate, and thus the subsequently obtained drivable area more accurate.

[0135] The following optional embodiment, in conjunction with the foregoing embodiment, describes the process of "step S104, mapping the non-ground point cloud to a pre-established target polar coordinate system, so as to determine the closest distance between the vehicle and the obstacles around the vehicle at all angles based on the non-ground point cloud in the target polar coordinate system".

[0136] The target polar coordinate system established in this step is as follows: Figure 3 As shown, the target polar coordinate system includes multiple angle grids, and each angle grid includes multiple radial grids. These angle grids are based on a preset angle resolution res. θ The resulting radial grid is based on a preset radial distance resolution res. dist The maximum distance between the target polar coordinate systems is obtained by partitioning. max .

[0137] Here, the maximum distance is determined by the hardware parameters of the lidar itself, typically set to 80m. This means the detection range extends within an 80m radius of the vehicle's rear axle center. Generally, the farther the lidar's detection range, the more likely the returned point cloud is noise. The maximum detection range of mechanical lidars is typically 100-150m. In this embodiment, the maximum distance is generally set to 80-100m. Taking a maximum vehicle speed of 120km / h on a highway as an example, environmental information can be predicted approximately 3 seconds in advance.

[0138] This optional embodiment can map the non-ground point cloud to a pre-established target polar coordinate system, and then determine the shortest distance from the vehicle to the obstacle within the 0-360° range of the vehicle based on the non-ground point cloud data in the target polar coordinate system.

[0139] Specifically, the process of "mapping the non-ground point set to a pre-established target polar coordinate system, and determining the closest distance between the vehicle and obstacles at all angles around the vehicle based on the non-ground point set in the target polar coordinate system" includes the following steps c1 to c3:

[0140] Step c1: Based on the x-coordinate and y-coordinate values ​​of each non-ground point cloud data in the non-ground point cloud set, calculate the horizontal angle and two-dimensional projection distance corresponding to each non-ground point cloud data.

[0141] Optionally, this step can be performed using the formula θ = tan θ, based on the x-coordinate and y-coordinate values ​​of each non-ground point cloud data point in the non-ground point cloud set. -1 (y / x) Calculate the horizontal angle θ corresponding to each non-ground point cloud data point, and use the formula Calculate the two-dimensional projection distance d corresponding to each non-ground point cloud data.

[0142] Step c2: Based on the horizontal angle and two-dimensional projection distance corresponding to each non-ground point cloud data in the non-ground point cloud set, map each non-ground point cloud data to the target polar coordinate system to obtain the non-ground point cloud set in the target polar coordinate system.

[0143] like Figure 3As shown, the target polar coordinate system contains many radial grids. This step can map each non-ground point cloud data contained in the non-ground point cloud set to the radial grids in the target polar coordinate system according to the horizontal angle θ and the vertical angle d.

[0144] Step c3: For each angle grid contained in the target polar coordinate system, determine the non-empty radial grid closest to the pole from the angle grid, calculate the average distance between the non-ground point cloud data contained in the non-empty radial grid and the pole, and use it as the closest distance between the obstacle and the vehicle at the corresponding angle of the angle grid; so as to obtain the closest distance between the obstacle and the vehicle at all angles around the vehicle.

[0145] In this step, the pole of the target polar coordinate system is located at the center of the vehicle's rear axle. After mapping all non-ground point cloud data to the target polar coordinate system in step c2, the non-empty radial grid closest to the pole (i.e., the center of the vehicle's rear axle) in each angle grid can be determined. Since the determined non-empty radial grid may contain multiple non-ground point cloud data, this step can calculate the average distance between the non-ground point cloud data contained in the non-empty radial grid and the pole, and use the determined average distance as the closest distance between the obstacle and the vehicle at the corresponding angle.

[0146] For example, angular grids within the 0–30° range, arranged in order of distance from the pole (radial grids 1, 2, 3, 4, and 5), contain radial grids 1 and 2 that do not contain non-ground point cloud data, while radial grids 3–5 do. Therefore, the non-empty radial grid closest to the pole within this 0–30° range is radial grid 3. Assuming radial grid 3 contains non-ground point cloud data 1–10, the average distance between non-ground point cloud data 1–10 and the pole can be used as the closest distance between the obstacle and the vehicle within this 0–30° angle range.

[0147] In this embodiment, the nearest distance is essentially the boundary information of the drivable area, that is, the distance to the nearest obstacle from the center of the vehicle's rear axle at a certain angle. This embodiment uses polar coordinates to express the nearest distance, improving the measurement accuracy to the centimeter level, making the calculation of the nearest distance more accurate. By calculating the nearest distance, the boundary of the drivable area can be determined, and a basis for subsequent calculations of the probability that each grid in the grid map contains an obstacle can be obtained, making the subsequently calculated probability more accurate.

[0148] In an optional embodiment, the process of "step S105, determining the probability that each grid in the preset grid map contains an obstacle based on the closest distance between the vehicle and the obstacles at all angles around the vehicle" will be described in conjunction with the foregoing embodiments.

[0149] Optionally, the process of "determining the probability that each grid in the preset grid map contains an obstacle based on the nearest distance between the vehicle and obstacles at all angles around the vehicle" may specifically include the following steps d1 to d2:

[0150] Step d1: Determine the probability that each grid cell in the preset grid map contains an obstacle based on the non-ground point cluster and the nearest distance.

[0151] In this step, a grid map can be set up in the area around the vehicle that can be collected by the LiDAR. The map is divided into several grids by specifying the single-frame map size (i.e., the size of the area collected by the LiDAR) and the grid resolution. Then, this step can map each non-ground point cloud contained in the non-ground point cloud into a grid to obtain the probability that each grid is occupied by an obstacle.

[0152] Considering that the nearest distance can characterize the boundary of the drivable area, the probability of a grid being occupied by an obstacle is low within the drivable area, and high outside the drivable area.

[0153] Therefore, when comprehensively considering the probability that each grid cell is occupied by an obstacle, in addition to considering the clustering of non-ground points, it is also necessary to consider the nearest distance, thereby obtaining the probability that each grid cell in the grid map contains an obstacle. Here, the probability that a grid cell contains an obstacle can characterize the probability that a vehicle can pass through that grid cell.

[0154] Step d2: Determine the drivable area of ​​the vehicle based on the probability that each grid in the grid map contains an obstacle.

[0155] This step determines the probability that each grid cell in the grid map contains an obstacle by determining the probability that non-ground point cloud data occupies a grid cell in the grid map.

[0156] Optionally, in conjunction with the foregoing embodiments, this step can determine the probability that each grid cell in the grid map contains an obstacle based on the non-ground point cloud, the target fitting plane equation, and the 0360° nearest distance. Here, the target fitting plane equation refers to the fitting plane equation determined in the last iteration.

[0157] Optionally, the specific process of "determining the probability that each grid cell in the grid map contains an obstacle based on the non-ground point cloud, the target fitted plane equation, and the nearest distance from 0 to 360°" includes the following steps d21 to d25:

[0158] Step d21: Based on the x-coordinate and y-coordinate values ​​of each non-ground point cloud data in the non-ground point cloud set and the preset grid resolution, map each non-ground point cloud data to the corresponding grid contained in the grid map.

[0159] Raster resolution is used to divide a raster map into several grids.

[0160] In this step, the coordinate transformation relationship Γ between the rear axis center and the origin of the raster map can be set, and each non-ground point cloud data in the non-ground point cloud set can be mapped to the corresponding raster contained in the raster map according to the coordinates (x,y) (it is worth noting that the non-ground point cloud data is data represented by three-dimensional coordinates, and here only the x-coordinate value and y-coordinate value in the three-dimensional coordinates are considered)

[0161] Specifically, using cell i,j Represents the raster in the i-th row and j-th column (i.e., the raster with coordinates (i,j)) of a raster map, denoted by res. cell Indicating raster resolution, when non-ground point cloud data p k (x k y k ) and cell i,j If the following relationship is satisfied, then p is considered to be... k ∈cell i,j .

[0162]

[0163] Step d22: Based on the target fitting plane equation and the preset second distance threshold, determine the non-ground point cloud data contained in the raster map as low point data or high point data.

[0164] Optionally, the specific implementation process of this step includes: for each non-ground point cloud data contained in the raster map (i.e., each non-ground point cloud data in the non-ground point cloud set), calculate the distance between the non-ground point cloud data and the target fitted plane equation. If the calculated distance is less than the second distance threshold, the non-ground point cloud data is determined as low point data; otherwise, the non-ground point cloud data is determined as high point data.

[0165] It should be noted that the second distance threshold in this step can be the same as or different from the first distance threshold mentioned above. The specific threshold needs to be determined according to the actual situation, and this application does not limit it.

[0166] Step d23: Based on the low point data and high point data contained in each grid cell in the grid map, as well as the low point data contained in adjacent grid cells, the preset grid occupancy probability, and the target fitting plane equation, determine the mixed occupancy probability corresponding to each grid cell.

[0167] Here, the preset grid occupancy probability refers to the probability that a grid cell will be occupied when a high-point data point is mapped onto it. For ease of subsequent description, this preset grid occupancy probability can be denoted as α.

[0168] In an optional embodiment, the step of "determining the mixed occupancy probability corresponding to each grid cell based on the low point data and high point data contained in each grid cell in the grid map, the low point data contained in adjacent grid cells, the preset grid occupancy probability, and the target fitting plane equation" may include the following steps d231 to d233:

[0169] Step d231: Calculate the number of high point data points contained in each grid cell in the grid map. Based on the number of high points and the preset grid cell occupancy probability, determine the high point occupancy probability corresponding to each grid cell in the grid map.

[0170] Optionally, for any grid cell contained in the raster map, if there are n high-point data points within that grid cell, then the probability that the high point corresponding to that grid cell is occupied is: P high =1-(1-α) n In the formula, P high refers to a grid cell i,j The corresponding probability of occupying the high point.

[0171] Step d232: Based on the low point data contained in each grid cell in the grid map, the low point data contained in adjacent grid cells, and the target fitted plane equation, determine the low point occupancy probability corresponding to each grid cell in the grid map.

[0172] Optionally, considering the accuracy of point cloud segmentation, the aforementioned non-ground point cloud set may not be very accurate. Thus, low-point data may be either non-ground point cloud data or ground point cloud data (i.e., low-point data is actually ground point cloud data, but this application identifies it as non-ground point cloud data). Therefore, in order to more accurately determine the low-point occupancy probability corresponding to the grid, the gradient change of the grid height can be used to characterize the low-point data's occupancy probability of the grid.

[0173] Based on this, the formula can be used. To determine the probability of low-point occupancy for each grid cell in a raster map, P is given by the formula. low refers to a grid cell i,j The corresponding low point occupancy probability, Represents a grid cell i,j The mean distance from all included low-point data points to the target fitted plane equation. Represents a grid cell k,l The mean distance from all low-point data to the target fitted plane equation is included, where (k,l)∈{(i-1,j-1),(i-1,j),(i-1,j+1),(i,j-1),(i,j+1),(i+1,j-1),(i+1,j),(i+1,j+1)}.

[0174] Step d233: Determine the mixed occupancy probability of each grid cell based on the high point occupancy probability and low point occupancy probability of each grid cell in the grid map.

[0175] It is understandable that there may be a grid that contains both high-point and low-point data. In this case, the mixed occupancy probability of the grid can be calculated in this step.

[0176] Optionally, it can be done using formula P. mix =1-(1-P) high (1-P) low ) to determine the raster cell i,j The corresponding mixed occupancy probability.

[0177] Step d24: Based on the coordinate transformation relationship between the rear axle center of the vehicle and the origin of the grid map, calculate the distance between each grid cell in the grid map and the rear axle center of the vehicle. Based on the calculated distance and the closest distance between the obstacle and the vehicle at the corresponding angle of each grid cell, determine the initial occupancy probability of each grid cell.

[0178] In this step, as described in the previous embodiments, the closest distance refers to the boundary of the drivable area. That is, for an angle grid, the map grid behind the closest distance between the obstacle and the vehicle at the corresponding angle is the drivable area.

[0179] Therefore, this step first calculates the distance d from the rear axle center of the vehicle to the vehicle's rear axle center based on the coordinate transformation relationship Γ between the vehicle's rear axle center and the origin of the grid map. tf Then, based on the aforementioned formula for calculating the horizontal angle θ, the value of each grid cell (e.g., cell) is calculated. i,j The corresponding horizontal angle θ is used to obtain the nearest distance for each grid cell.

[0180] Following this, this step can be based on the distance d from each grid cell to the center of the vehicle's rear axle. tf The nearest distance to each grid cell The initial occupancy probability for each grid cell in the raster map is calculated using the following formula.

[0181]

[0182] In the formula, P init refers to a grid cell i,j The corresponding initial occupancy probability.

[0183] Step d25: Determine the probability that each grid in the grid map contains an obstacle based on the mixed occupancy probability and the initial occupancy probability corresponding to each grid in the grid map.

[0184] This step combines the mixed occupancy probability and the initial occupancy probability to obtain the final occupancy probability, which is the probability that the grid contains an obstacle.

[0185] Optionally, it can be done using formula P. final =1-(1-P) mix (1-P) init ) to determine the raster cell i,j The probability of containing obstacles.

[0186] It should be noted that the higher the probability value determined in this step, the higher the probability that the grid is occupied by an obstacle and the lower the probability that it is passable.

[0187] In this embodiment, the area inside the boundary of the drivable area is quantitatively evaluated by using a grid map. The entire detection range is gridded, and a probability value is used to characterize the drivability probability of the grid contained in the grid map, resulting in a more accurate calculated probability.

[0188] In summary, this application provides a method for detecting drivable areas for intelligent driving based on multiple lidar sensors. Compared with target detection methods based on cameras, the detection distance is greatly improved and the environmental adaptability is good. Compared with detection methods based on lidar and high-precision maps, it does not rely on road information. In other words, this application only relies on lidar perception information and does not require additional conditions such as high-precision maps and structured roads, resulting in high autonomous detection performance.

[0189] Furthermore, by supplementing the vehicle's left and right detection blind spots with left and right radars, the technical solution utilizes features such as closest distance and occupancy probability maps. If only the main LiDAR is used, especially in scenarios such as automatic parking, the main radar will have blind spots. The calculation of closest distance and occupancy probability on the left and right sides will ignore obstacles in the blind spots. Therefore, on this basis, the left and right radars are used to reduce perception blind spots. Intuitively speaking, the supplementation of left and right radars will result in more point cloud information and higher point cloud density on the left and right sides of the vehicle.

[0190] From the perspective of drivable area calculation results, the calculation of nearest distance and occupied grid map will be more accurate; by adopting the nearest distance expression method of polar coordinates, the measurement accuracy of the nearest distance is improved to the centimeter level; finally, the rasterization of single frame point cloud is realized, which can serve the path planning and control of intelligent driving.

[0191] This application also provides an intelligent driving drivable area detection device. The intelligent driving drivable area detection device provided in this application is described below. The intelligent driving drivable area detection device described below and the intelligent driving drivable area detection method described above can be referred to and correspond to each other.

[0192] Please see Figure 4 The diagram shows a structural schematic of the intelligent driving drivable area detection device provided in an embodiment of this application. Figure 4 As shown, the intelligent driving drivable area detection device may include: a point cloud data acquisition module 401, a point cloud data fusion module 402, a non-ground point cloud set determination module 403, a nearest distance determination module 404, and a drivable area determination module 405.

[0193] The point cloud data acquisition module 401 is used to acquire point cloud data from multiple lidars at the same timestamp. The point cloud data is collected by lidars installed on the vehicle, and all point cloud data are transformed into the vehicle coordinate system. The point cloud data includes multiple frames of point cloud data.

[0194] The point cloud data fusion module 402 is used to fuse multiple frames of point cloud data to obtain a fused point cloud set.

[0195] The non-ground point cloud set determination module 403 is used to determine non-ground point cloud data from the fused point cloud set based on the z-coordinate values ​​of the fused point cloud data contained in the fused point cloud set, so as to obtain a non-ground point cloud set composed of the determined non-ground point cloud data.

[0196] The closest distance determination module 404 is used to map the non-ground point cloud to a pre-established target polar coordinate system, so as to determine the closest distance between the vehicle and the obstacles around the vehicle at all angles based on the non-ground point cloud in the target polar coordinate system. The target polar coordinate system is a polar coordinate system established with the origin of the vehicle coordinate system as the pole.

[0197] The drivable area determination module 405 is used to determine the drivable area of ​​the vehicle based on the closest distance between the vehicle and obstacles around the vehicle at all angles.

[0198] The intelligent driving drivable area detection device provided in this application first acquires point cloud data from multiple LiDARs at the same timestamp. Then, it fuses multiple frames of point cloud data to obtain a fused point cloud set. Next, based on the z-coordinate values ​​of the fused point cloud data contained in the fused point cloud set, it determines non-ground point cloud data from the fused point cloud set to obtain a non-ground point cloud set composed of the determined non-ground point cloud data. Then, it maps the non-ground point cloud set to a pre-established target polar coordinate system. Based on the non-ground point cloud set in the target polar coordinate system, it determines the closest distances between obstacles around the vehicle at all angles. Finally, based on the closest distances between obstacles around the vehicle at all angles, it determines the drivable area of ​​the vehicle. This application uses multiple LiDARs to collect point cloud data, significantly increasing the detection range and improving environmental adaptability. Furthermore, this application can determine the closest distances between obstacles around the vehicle at all angles, accurately obtaining the boundaries of the drivable area around the vehicle at each angle based on the closest distances. The entire process relies solely on the point cloud data collected by the LiDARs, requiring no additional conditions such as high-precision maps or structured roads, resulting in high autonomous detection performance.

[0199] In one possible implementation, the point cloud data fusion module 402 can be used to stitch multiple frames of point cloud data into a single frame of point cloud data according to a preset lidar sequence to obtain a fused point cloud set. Alternatively, for each point cloud data contained in the multiple frames of point cloud data, the module can calculate the horizontal and vertical angles corresponding to that point cloud data and fill the point cloud data into the corresponding point cloud grid based on the horizontal and vertical angles. This ensures that each point cloud data contained in the multiple frames of point cloud data is filled into the corresponding point cloud grid. For each point cloud grid, if the grid contains multiple point cloud data, the centroid of the multiple point cloud data is used as the fused point cloud data under that grid. This results in a fused point cloud set composed of the fused point cloud data from all the point cloud grids.

[0200] In one possible implementation, the above-mentioned non-ground point cloud set determination module 403 may include: an abnormal point cloud removal module, a ground point cloud set determination module to be updated, a fitted plane equation determination module, an updated ground point cloud set determination module, and an update loop module.

[0201] The abnormal point cloud removal module is used to remove the abnormal z-coordinate values ​​in the fused point cloud set and use the fused point cloud set after removing the abnormal data as the target fused point cloud set.

[0202] The ground point cloud set to be updated module is used to determine ground point cloud data from the target fused point cloud set based on the z-coordinate values ​​of the fused point cloud data contained in the target fused point cloud set, so as to obtain the ground point cloud set to be updated composed of the determined ground point cloud data.

[0203] The module for determining the fitted plane equation is used to determine the fitted plane equation based on the ground point cloud set to be updated.

[0204] The updated ground point cloud set determination module is used to calculate the distance between each fused point cloud data in the target fused point cloud set and the fitted plane equation. The fused point cloud data whose calculated distance is less than a preset first distance threshold is determined as ground point cloud data, and the determined ground point cloud data is used as the updated ground point cloud set.

[0205] The update loop module increments the update iteration count by 1 and determines whether the incremented update iteration count has reached the preset total number of iterations. If not, the updated ground point cloud set is used as the ground point cloud data to be updated, and the process returns to the fitting plane equation determination module until the update iteration count reaches the total number of iterations. Then, the other point cloud data in the target fused point cloud set, excluding the updated ground point cloud set, are used as non-ground point cloud data. The initial update iteration count is 0.

[0206] In one possible implementation, the above-mentioned ground point cloud set determination module may include: a z-axis average value calculation module and a difference comparison module.

[0207] The z-axis average value calculation module is used to select a preset number of fused point cloud data from the target fused point cloud set based on the z-axis coordinate values ​​of the fused point cloud data contained in the target fused point cloud set, and calculate the z-axis average value of the preset number of fused point cloud data.

[0208] The difference comparison module is used to determine that the fused point cloud data is ground point cloud data if the difference between the z-coordinate value and the z-average value of each fused point cloud data in the target fused point cloud set is less than a preset difference threshold.

[0209] In one possible implementation, the above-mentioned fitting plane equation determination module may include: a three-dimensional average calculation module, a feature information solution module, a normal vector determination module, and a fitting plane equation solution module.

[0210] The three-axis average calculation module is used to calculate the x-axis average, y-axis average, and z-axis average of the ground point cloud to be updated.

[0211] The feature information solving module is used to determine the covariance matrix based on the x-axis average, y-axis average and z-axis average of the ground point cloud to be updated, and to solve for several eigenvalues ​​and eigenvectors of the covariance matrix.

[0212] The normal vector determination module is used to determine the smallest eigenvalue from several eigenvalues, and the eigenvector corresponding to the smallest eigenvalue is used as the normal vector of the fitting plane.

[0213] The module for solving the fitted plane equation is used to solve for the fitted plane equation based on the ground point cloud set to be updated and the normal vector of the fitted plane.

[0214] In one possible implementation, the target polar coordinate system includes multiple angular grids, and each angular grid includes multiple radial grids.

[0215] Based on this, the aforementioned nearest distance determination module 403 may include: an angle distance calculation module, a first point cloud mapping module, and a distance mean calculation module.

[0216] The angle and distance calculation module is used to calculate the horizontal angle and two-dimensional projection distance of each non-ground point cloud data based on the x-coordinate and y-coordinate values ​​of each non-ground point cloud data in the non-ground point cloud set.

[0217] The first point cloud mapping module is used to map each non-ground point cloud data to the target polar coordinate system based on the horizontal angle and two-dimensional projection distance corresponding to each non-ground point cloud data in the non-ground point cloud set, so as to obtain the non-ground point cloud set in the target polar coordinate system.

[0218] The distance mean calculation module is used to determine the non-empty radial grid closest to the pole from each angle grid contained in the target polar coordinate system, and calculate the distance mean between the non-ground point cloud data contained in the non-empty radial grid and the pole, which is used as the closest distance between the obstacle and the vehicle at the corresponding angle of the angle grid; so as to obtain the closest distance between the obstacle and the vehicle at all angles around the vehicle.

[0219] In one possible implementation, the aforementioned drivable area determination module 404 may include: an occupancy probability determination module and an occupancy probability application module.

[0220] The occupancy probability determination module is used to determine the probability that each grid in the preset grid map contains an obstacle based on the non-ground point cluster and the nearest distance.

[0221] The occupancy probability application module is used to determine the drivable area of ​​a vehicle based on the probability that each grid in the grid map contains an obstacle.

[0222] In one possible implementation, the above-mentioned occupancy probability determination module may include: a second point cloud mapping module, a high and low point data determination module, a mixed occupancy probability determination module, an initial occupancy probability determination module, and a final occupancy probability determination module.

[0223] The second point cloud mapping module is used to map each non-ground point cloud data into the corresponding grid contained in the grid map based on the x-coordinate value and y-coordinate value of each non-ground point cloud data in the non-ground point cloud set and the preset grid resolution. The grid resolution is used to divide the grid map into several grids.

[0224] The high and low point data determination module is used to determine the non-ground point cloud data contained in the raster map as low point data or high point data according to the target fitting plane equation and the preset second distance threshold. The target fitting plane equation is the fitting plane equation determined in the last iteration.

[0225] The mixed occupancy probability determination module is used to determine the mixed occupancy probability of each grid cell based on the low point data and high point data contained in each grid cell in the grid map, the low point data contained in adjacent grid cells, the preset grid occupancy probability, and the target fitting plane equation.

[0226] The initial occupancy probability determination module is used to calculate the distance between each grid in the grid map and the rear axle center of the vehicle based on the coordinate transformation relationship between the rear axle center of the vehicle and the origin of the grid map. Based on the calculated distance and the closest distance between the obstacle and the vehicle at the corresponding angle of each grid, the initial occupancy probability of each grid is determined.

[0227] The final occupancy probability determination module is used to determine the probability that each grid in the grid map contains an obstacle based on the mixed occupancy probability and the initial occupancy probability corresponding to each grid in the grid map.

[0228] In one possible implementation, the above-mentioned hybrid occupancy probability determination module may include: a high point occupancy probability determination module, a low point occupancy probability determination module, and a hybrid occupancy probability calculation module.

[0229] The high point occupancy probability determination module is used to calculate the number of high point data contained in each grid cell in the grid map, and determine the high point occupancy probability corresponding to each grid cell in the grid map based on the number of high points and the preset grid occupancy probability.

[0230] The low-point occupancy probability determination module is used to determine the low-point occupancy probability of each grid in the grid map based on the low-point data contained in each grid in the grid map, the low-point data contained in adjacent grids, and the target fitted plane equation.

[0231] The mixed occupancy probability calculation module is used to determine the mixed occupancy probability of each grid cell based on the high point occupancy probability and low point occupancy probability of each grid cell in the grid map.

[0232] This application also provides an intelligent driving drivable area detection device. Optionally, Figure 5The hardware structure block diagram of the intelligent driving drivable area detection device is shown below. Figure 5 The hardware structure of the intelligent driving drivable area detection device may include: at least one processor 501, at least one communication interface 502, at least one memory 503 and at least one communication bus 504.

[0233] In this embodiment of the application, the number of processor 501, communication interface 502, memory 503 and communication bus 504 is at least one, and processor 501, communication interface 502 and memory 503 communicate with each other through communication bus 504.

[0234] The processor 501 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention.

[0235] The memory 503 may include high-speed RAM, and may also include non-volatile memory, such as at least one disk storage device;

[0236] The memory 503 stores a program, and the processor 501 can call the program stored in the memory 503. The program is used for:

[0237] Point cloud data at the same timestamp from multiple lidar sensors is acquired. The point cloud data is collected by lidar sensors installed on the vehicle, and all point cloud data is transformed to the vehicle coordinate system. The point cloud data includes multiple frames of point cloud data.

[0238] Multi-frame point cloud data are fused to obtain a fused point cloud set;

[0239] Based on the z-coordinate values ​​of the fused point cloud data contained in the fused point cloud set, non-ground point cloud data is determined from the fused point cloud set to obtain a non-ground point cloud set composed of the determined non-ground point cloud data;

[0240] The non-ground point cloud is mapped to a pre-established target polar coordinate system. Based on the non-ground point cloud in the target polar coordinate system, the closest distance between the vehicle and obstacles around the vehicle at all angles is determined. The target polar coordinate system is a polar coordinate system established with the origin of the vehicle coordinate system as the pole.

[0241] The drivable area of ​​the vehicle is determined by the closest distance between the vehicle and obstacles at all angles around it.

[0242] Optionally, the refined and extended functions of the program can be found in the description above.

[0243] This application also provides a readable storage medium storing a computer program thereon, which, when executed by a processor, implements the intelligent driving drivable area detection method described above.

[0244] Optionally, the refined and extended functions of the program can be found in the description above.

[0245] Finally, it should be noted that in this document, relational terms such as "second" and "etc." are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0246] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0247] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for detecting drivable areas in intelligent driving, characterized in that, include: Point cloud data at the same timestamp from multiple lidar sensors are acquired. The point cloud data is collected by lidar sensors installed on the vehicle, and all point cloud data are transformed to the vehicle coordinate system. The point cloud data includes multiple frames of point cloud data. The multi-frame point cloud data is fused to obtain a fused point cloud set; Based on the z-coordinate values ​​of the fused point cloud data contained in the fused point cloud set, non-ground point cloud data is determined from the fused point cloud set to obtain a non-ground point cloud set composed of the determined non-ground point cloud data. The non-ground point cloud is mapped to a pre-established target polar coordinate system, so as to determine the closest distance between the vehicle and the obstacles around the vehicle at all angles based on the non-ground point cloud under the target polar coordinate system. The target polar coordinate system is a polar coordinate system established with the origin of the vehicle coordinate system as the pole. The drivable area of ​​the vehicle is determined based on the nearest distances between the vehicle and obstacles around it at all angles, including: Based on the x-coordinate and y-coordinate values ​​of each non-ground point cloud data in the non-ground point cloud set and the preset grid resolution, each non-ground point cloud data is mapped to the corresponding grid contained in the grid map, wherein the grid resolution is used to divide the grid map into several grids. Based on the target fitting plane equation and the preset second distance threshold, the non-ground point cloud data contained in the raster map is determined as low point data or high point data, wherein the target fitting plane equation is the fitting plane equation determined in the last iteration; Based on the low point data and high point data contained in each grid cell in the grid map, as well as the low point data contained in adjacent grid cells, the preset grid occupancy probability, and the target fitting plane equation, the mixed occupancy probability corresponding to each grid cell is determined. Based on the coordinate transformation relationship between the vehicle's rear axle center and the origin of the grid map, the distance between each grid cell in the grid map and the vehicle's rear axle center is calculated. Based on the calculated distances and the nearest distance between the vehicle and an obstacle at the corresponding angle for each grid cell, the initial occupancy probability for each grid cell is determined. Specifically, based on the distance between each grid cell and the vehicle's rear axle center and the nearest distance for each grid cell, the initial occupancy probability for each grid cell is calculated using the following formula: In the formula, This refers to the initial occupancy probability of the grid. The distance from each grid cell to the center of the vehicle's rear axle. The nearest distance for each grid cell; Based on the mixed occupancy probability and initial occupancy probability corresponding to each grid cell in the grid map, determine the probability that each grid cell in the grid map contains an obstacle; The drivable area of ​​the vehicle is determined based on the probability that each grid cell in the grid map contains an obstacle.

2. The intelligent driving drivable area detection method according to claim 1, characterized in that, The process of fusing the multi-frame point cloud data to obtain a fused point cloud set includes: The multi-frame point cloud data is stitched together into a single frame of point cloud data according to a preset lidar sequence to obtain the fused point cloud set; or, For each point cloud data contained in the multi-frame point cloud data, calculate the horizontal and vertical angles corresponding to the point cloud data, and fill the point cloud data into the corresponding point cloud grid according to the horizontal and vertical angles corresponding to the point cloud data; so that each point cloud data contained in the multi-frame point cloud data is filled into the corresponding point cloud grid. For each point cloud grid, if the point cloud grid contains multiple point cloud data, the centroid of the multiple point cloud data is taken as the fused point cloud data under the point cloud grid; so as to obtain a fused point cloud set composed of the fused point cloud data under all point cloud grids.

3. The intelligent driving drivable area detection method according to claim 1, characterized in that, The step of determining non-ground point cloud data from the fused point cloud set based on the z-coordinate values ​​of the fused point cloud data included in the fused point cloud set includes: Remove the fused point cloud data containing the abnormal z-coordinate values ​​in the fused point cloud set, and take the fused point cloud set after removing the abnormal data as the target fused point cloud set; Based on the z-coordinate values ​​of the fused point cloud data contained in the target fused point cloud set, ground point cloud data is determined from the target fused point cloud set to obtain a ground point cloud set to be updated composed of the determined ground point cloud data. Determine the fitted plane equation based on the ground point cloud set to be updated; Calculate the distance between each fused point cloud data in the target fused point cloud set and the fitted plane equation, and determine the fused point cloud data whose calculated distance is less than a preset first distance threshold as ground point cloud data. Use the determined ground point cloud data as the updated ground point cloud set. Increment the update iteration count by 1, and determine whether the updated iteration count after incrementing by 1 has reached the preset total number of iterations. If not, use the updated ground point cloud set as the ground point cloud set to be updated, and return to execute the step of determining the fitting plane equation based on the ground point cloud set to be updated, until the update iteration count reaches the preset total number of iterations. Then, use the other point cloud data in the target fused point cloud set other than the updated ground point cloud set as non-ground point cloud data. The initial update iteration count is 0.

4. The intelligent driving drivable area detection method according to claim 3, characterized in that, The step of determining ground point cloud data from the target fused point cloud set based on the z-coordinate values ​​of the fused point cloud data contained in the target fused point cloud set includes: Based on the z-coordinate values ​​of the fused point cloud data contained in the target fused point cloud set, a preset number of fused point cloud data are selected from the target fused point cloud set, and the average z-coordinate value of the preset number of fused point cloud data is calculated. For each fused point cloud data in the target fused point cloud set, if the difference between the z-axis coordinate value of the fused point cloud data and the z-axis average value is less than a preset difference threshold, then the fused point cloud data is determined to be ground point cloud data.

5. The intelligent driving drivable area detection method according to claim 3, characterized in that, Determine the fitted plane equation based on the ground point cloud set to be updated, including: Calculate the x-axis average, y-axis average, and z-axis average of the ground point cloud to be updated; Based on the x-axis average, y-axis average, and z-axis average of the ground point cloud to be updated, the covariance matrix is ​​determined, and several eigenvalues ​​and eigenvectors of the covariance matrix are solved. The smallest eigenvalue is determined from the plurality of eigenvalues, and the eigenvector corresponding to the smallest eigenvalue is used as the normal vector of the fitting plane; The equation of the fitting plane is solved based on the ground point cloud set to be updated and the normal vector of the fitting plane.

6. The intelligent driving drivable area detection method according to claim 3, characterized in that, The target polar coordinate system includes multiple angular grids, and each angular grid includes multiple radial grids; The step of mapping the non-ground point set to a pre-established target polar coordinate system, and determining the closest distance between the vehicle and obstacles around the vehicle at all angles based on the non-ground point set in the target polar coordinate system, includes: Based on the x-coordinate and y-coordinate values ​​of each non-ground point cloud data in the non-ground point cloud set, the horizontal angle and two-dimensional projection distance corresponding to each non-ground point cloud data are calculated. Based on the horizontal angle and two-dimensional projection distance corresponding to each non-ground point cloud data in the non-ground point cloud set, each non-ground point cloud data is mapped to the target polar coordinate system to obtain the non-ground point cloud set in the target polar coordinate system. For each angle grid contained in the target polar coordinate system, the non-empty radial grid closest to the pole is determined from the angle grid, and the average distance between the non-ground point cloud data contained in the non-empty radial grid and the pole is calculated as the closest distance between the obstacle and the vehicle at the corresponding angle of the angle grid; so as to obtain the closest distance between the obstacle and the vehicle at all angles around the vehicle.

7. The intelligent driving drivable area detection method according to claim 1, characterized in that, The step of determining the mixed occupancy probability of each grid cell based on the low-point and high-point data contained in each grid cell of the grid map, the low-point data contained in adjacent grid cells, the preset grid occupancy probability, and the target fitting plane equation includes: Calculate the number of high-point data points contained in each grid cell of the grid map, and determine the high-point occupancy probability corresponding to each grid cell based on the number of high-point data points and the preset grid cell occupancy probability. Based on the low point data contained in each grid cell of the grid map, the low point data contained in adjacent grid cells, and the target fitting plane equation, the low point occupancy probability corresponding to each grid cell in the grid map is determined. Based on the high point occupancy probability and low point occupancy probability of each grid in the grid map, determine the mixed occupancy probability of each grid.

8. A device for detecting drivable areas for intelligent driving, characterized in that, include: The point cloud data acquisition module is used to acquire point cloud data from multiple lidar sensors at the same timestamp. The point cloud data is collected by lidar sensors installed on the vehicle, and all point cloud data is transformed to the vehicle coordinate system. The point cloud data includes multiple frames of point cloud data. The point cloud data fusion module is used to fuse the multi-frame point cloud data to obtain a fused point cloud set; The non-ground point cloud set determination module is used to determine non-ground point cloud data from the fused point cloud set based on the z-coordinate values ​​of the fused point cloud data contained in the fused point cloud set, so as to obtain a non-ground point cloud set composed of the determined non-ground point cloud data. The closest distance determination module is used to map the non-ground point set to a pre-established target polar coordinate system, so as to determine the closest distance between the vehicle and the obstacles around the vehicle at all angles according to the non-ground point set under the target polar coordinate system. The target polar coordinate system is a polar coordinate system established with the origin of the vehicle coordinate system as the pole. The drivable area determination module is used to determine the drivable area of ​​the vehicle based on the closest distance between the vehicle and obstacles around the vehicle from all angles. The drivable area determination module is specifically used for: Based on the x-coordinate and y-coordinate values ​​of each non-ground point cloud data in the non-ground point cloud set and the preset grid resolution, each non-ground point cloud data is mapped to the corresponding grid contained in the grid map, wherein the grid resolution is used to divide the grid map into several grids. Based on the target fitting plane equation and the preset second distance threshold, the non-ground point cloud data contained in the raster map is determined as low point data or high point data, wherein the target fitting plane equation is the fitting plane equation determined in the last iteration; Based on the low point data and high point data contained in each grid cell in the grid map, as well as the low point data contained in adjacent grid cells, the preset grid occupancy probability, and the target fitting plane equation, the mixed occupancy probability corresponding to each grid cell is determined. Based on the coordinate transformation relationship between the vehicle's rear axle center and the origin of the grid map, the distance between each grid cell in the grid map and the vehicle's rear axle center is calculated. Based on the calculated distances and the nearest distance between the vehicle and an obstacle at the corresponding angle for each grid cell, the initial occupancy probability for each grid cell is determined. Specifically, based on the distance between each grid cell and the vehicle's rear axle center and the nearest distance for each grid cell, the initial occupancy probability for each grid cell is calculated using the following formula: In the formula, This refers to the initial occupancy probability of the grid. The distance from each grid cell to the center of the vehicle's rear axle. The nearest distance for each grid cell; Based on the mixed occupancy probability and initial occupancy probability corresponding to each grid cell in the grid map, determine the probability that each grid cell in the grid map contains an obstacle; The drivable area of ​​the vehicle is determined based on the probability that each grid cell in the grid map contains an obstacle.

Images