Method for detecting drivable area of self-driving automobile

Abstract

The invention discloses a method for detecting a drivable area of an autonomous vehicle. The method comprises the following steps: 1) collecting point cloud data and image data; 2) calculating the average brightness of the image data, and judging whether the average brightness of the image data is greater than an average brightness threshold value or not; 3) if the brightness is greater than the average brightness threshold, inputting the image data into a semantic segmentation model, outputting confidence by the semantic segmentation model, and executing a step 4), otherwise, calculating the number of point clouds of the point cloud data, and executing a step 5); 4) judging whether the confidence meets the requirement of outputting a drivable area, and if so, outputting a grid map of the drivable area by the semantic segmentation model; if not, calculating the point cloud number of the point cloud data, and executing the step 5); (5) if the number of the point clouds is smaller than 5000, automatic driving is ended; otherwise, processing the point cloud data through a plane extraction method, a nine-grid domain method and an image domain method in sequence, mapping the drivable area point cloud into the grid map, and outputting the grid map of the drivable area.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving, and in particular to a method for detecting a drivable area of ​​an autonomous driving vehicle. Background Art

[0002] In order for an autonomous vehicle to be able to drive, it must detect obstacles in the current environment and find the drivable area before it can avoid obstacles and drive automatically. Drivable area detection is relatively easy for structured roads, because structured roads are relatively flat and easy to detect planes; however, for unstructured environments or structured scenes with harsh environments, the ground is rugged and uneven, which undoubtedly increases the difficulty of drivable area detection.

[0003] In the prior art, two methods are usually used to detect drivable areas: (1) Using a deep learning semantic segmentation model. This method has good detection effect and high precision, but it is extremely dependent on the training data set when building the model, which means that more time and financial resources need to be invested in data collection, cleaning, labeling, training, etc. If the initial investment in the data set is insufficient, it is easy to mistakenly detect background obstacles as drivable areas. In addition, the sparsity of point clouds in different perception ranges varies. Sparse point clouds at a distance are a great challenge to deep learning, so only dense point clouds at a close distance are suitable for deep learning semantic segmentation. (2) Using geometric algorithms, such as the plane extraction method, this method is simple to calculate, efficient, easy to implement, and can quickly extract boundaries and features. It is suitable for real-time processing and low-complexity scenes. However, this method is easily affected by noise and has poor robustness to occlusion and non-uniform illumination, which limits its application effect in complex scenes. Summary of the invention

[0004] In view of the above problems, the present invention provides a method for detecting the drivable area of ​​an autonomous vehicle, which combines the semantic segmentation model with the geometric algorithm, and combines the perception advantages of the two algorithms in the local information field and the global information field, which can not only improve the detection accuracy of the drivable area, but also reduce the missed detection rate of background obstacles. The algorithm also combines the advantages of images and point clouds, and this cross-modal post-fusion can achieve a redundant effect.

[0005] The technical solution of the present invention is: a method for detecting a drivable area of ​​an autonomous driving vehicle, comprising the following steps:

[0006] 1) Collect point cloud data through the laser radar installed on the self-driving car, and collect image data through the surround camera;

[0007] 2) Calculate the average brightness of the image data, and determine whether the average brightness of the image data is greater than the average brightness threshold;

[0008] 3) If the average brightness of the image data is greater than the average brightness threshold, the image data is input into the semantic segmentation model, the semantic segmentation model outputs the confidence, and step 4) is executed; otherwise, the number of point clouds of the point cloud data is calculated, and step 5) is executed;

[0009] 4) Determine whether the confidence level satisfies the output of the drivable area. If so, the semantic segmentation model outputs a grid map of the drivable area, and the autonomous vehicle drives according to the grid map. If not, calculate the number of point cloud data points and execute step 5).

[0010] 5) If the number of point clouds is less than 5000, the autonomous driving car will stop directly and end the automatic driving; otherwise, the point cloud data will be processed by the plane extraction method, the nine-square field method, and the image field method in turn, and the point cloud of the drivable area in the point cloud data will be output, and the point cloud of the drivable area will be mapped to the raster map, and the raster map of the drivable area will be output. The autonomous driving car will drive according to the raster map.

[0011] Preferably, in step 1), the collected point cloud data and image data also need to be preprocessed, wherein the point cloud data is subjected to denoising and filtering, and the image data is subjected to distortion correction, color normalization, and denoising.

[0012] Preferably, in step 1), the timestamps of the laser radar and the surround view camera are based on the same time reference.

[0013] Preferably, the plane extraction method extracts planes from the initial drivable point cloud using a RANSAC algorithm.

[0014] Preferably, the image domain method converts the point cloud coordinates into image coordinates through a coordinate conversion formula, and the coordinate conversion formula is:

[0015]

[0016] Where [w,h] is the row and column of the image coordinates, [x,y,z] is the point cloud coordinates, θg is the horizontal and vertical resolution of the lidar, and floor is rounded up.

[0017] The advantages of the present invention are:

[0018] 1. The present invention greatly reduces the amount of data for training the semantic segmentation model. By combining the semantic segmentation model with the geometric algorithm, the geometric algorithm is used to supplement and improve the detection accuracy of the semantic segmentation model, thereby reducing the time and cost of training the semantic segmentation model.

[0019] 2. The present invention combines two algorithms to improve the perception of the present invention in the local information field and the global information field, greatly increase the detection accuracy of the executable area, and reduce the missed detection rate of background obstacles. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 It is a flow chart of the present invention. DETAILED DESCRIPTION

[0021] See also Figure 1 , a method for detecting a drivable area of ​​an autonomous driving vehicle, comprising the following steps:

[0022] 1) The laser radar installed on the self-driving car collects point cloud data, and the surround camera collects image data. After collecting the point cloud data and image data, preprocessing is required, in which the point cloud data is denoised and filtered, and the image data is distorted, color normalized, and denoised to reduce noise interference and improve data quality. The timestamps of the laser radar and the surround camera are based on the same time reference. By adjusting the hardware of the laser radar and the surround camera, the time between the sub-processing module and the core algorithm module of the laser radar and the surround camera is accurate. In this example, the time of the laser radar and the surround camera is synchronized to the microsecond level.

[0023] 2) Calculate the average brightness of the image data and determine whether the average brightness of the image data is greater than an average brightness threshold. In this embodiment, the average brightness threshold is set to 100;

[0024] 3) If the average brightness of the image data is greater than 100, the image data is input into the semantic segmentation model, the semantic segmentation model outputs the confidence, and step 4) is executed; otherwise, the number of point clouds of the point cloud data is calculated, and step 5) is executed;

[0025] 4) Determine whether the confidence level satisfies the output of the drivable area. If so, the semantic segmentation model outputs a grid map of the drivable area, and the autonomous vehicle drives according to the grid map. If not, calculate the number of point cloud data points and execute step 5).

[0026] 5) If the number of point clouds is less than 5000, the autonomous driving car will stop directly and end the automatic driving; otherwise, the point cloud data will be processed by the plane extraction method, the nine-square field method, and the image field method in turn, and the point cloud of the drivable area in the point cloud data will be output, and the point cloud of the drivable area will be mapped to the raster map, and the raster map of the drivable area will be output. The autonomous driving car will drive according to the raster map.

[0027] Since operations such as convolution and pooling in the semantic segmentation model will increase its perception field of view, its perception effect of global information is better than that of geometric algorithms, but this will sacrifice the accuracy of some local information, resulting in some small obstacles being mistakenly identified as drivable areas. Therefore, it is necessary to extract information with higher confidence in the semantic information as the initial drivable point cloud. The plane extraction method uses the RANSAC algorithm to extract the plane of the initial drivable point cloud. The RANSAC algorithm can further increase the confidence of the point cloud. The plane model of the plane extraction method is:

[0028] Ax+By+Cz+D=0

[0029] Where [x, y, z] is the point cloud coordinates, and A, B, C, and D are all parameters to be determined.

[0030] A plane can be determined by three point cloud coordinates. The normal vectors formed between the three point cloud coordinates are calculated, and the angle between the normal vector of the plane and the z-axis is calculated to obtain the slope size at that location. The plane with too large a slope is eliminated, and the point cloud close to the fitting plane is found, which is the point cloud of the drivable area.

[0031] Since the actual road conditions of the autonomous driving car are complicated, the point cloud that meets the requirements of the plane extraction method only accounts for a small part of the total point cloud data. Therefore, the Nine Palaces Domain Method further extracts the drivable area point cloud based on the plane extraction method, and finds the missed point cloud through the area expansion algorithm of the Nine Palaces Domain Method. The Nine Palaces Domain Method first processes the total point cloud data into a grid table form, and sets a 3×3 search window, which contains the drivable area point cloud and the point cloud to be detected in the search window. If the point cloud to be detected in the search window meets a certain geometric relationship with the drivable area point cloud, such as the height is similar or the angle with the center point is similar, then the part of the point cloud to be detected can be judged as a new drivable area point cloud. In this embodiment, whether the point cloud to be detected is close to the drivable area point cloud is used to judge whether the point cloud to be detected is a drivable area point cloud. If the point cloud to be detected is close to the drivable area point cloud, then the point cloud to be detected is a drivable area point cloud, otherwise, it is not a drivable area point cloud.

[0032] Similarly, in order to ensure that the point cloud of the drivable area is not missed as much as possible, after the nine-square field method, the remaining point cloud needs to be further judged by the image field method. The image field method converts the point cloud coordinates into image coordinates. A 3×3 search window is also set. The search window contains the drivable area point cloud and the image pixels of the point cloud to be detected. If the drivable area point cloud and the image pixels of the point cloud to be detected in the search window meet certain geometric relationships, such as similar heights or similar angles with the center point, the image pixels of the point cloud to be detected in this part are judged as new drivable area point cloud image pixels. The image field method converts the point cloud coordinates into image coordinates through the coordinate conversion formula. The coordinate conversion formula is:

[0033]

[0034] Where [w,h] is the row and column of the image coordinates, [x,y,z] is the point cloud coordinates, θg is the horizontal and vertical resolution of the lidar, and floor is rounded up.

[0035] The present invention supplements the semantic segmentation model through the geometric algorithm, which can reduce the amount of data for training the semantic segmentation model and greatly reduce the training time and training cost of the semantic segmentation model. Although the detection accuracy of the semantic segmentation model is slightly reduced, the detection accuracy of the semantic segmentation model is supplemented and improved by the geometric algorithm, which ensures that the detection accuracy of the entire algorithm is high and the missed detection rate is low.

Claims

1. A method for detecting a driving area of ​​an autonomous vehicle, characterized in that: The following steps are involved: 1) The laser radar installed on the self-driving car collects point cloud data, and the surround camera collects image data; 2) Calculate the average brightness of the image data, and determine whether the average brightness of the image data is greater than the average brightness threshold; 3) If the average brightness of the image data is greater than the average brightness threshold, the image data is input into the semantic segmentation model, the semantic segmentation model outputs the confidence, and step 4) is executed; otherwise, the number of point clouds of the point cloud data is calculated, and step 5) is executed; 4) Determine whether the confidence level meets the requirement to output the drivable area. If so, the semantic segmentation model outputs a grid map of the drivable area, and the autonomous vehicle drives according to the grid map. If not satisfied, calculate the number of point cloud data points and execute step 5); 5) If the number of point clouds is less than 5000, the autonomous driving car will stop directly and end the automatic driving; otherwise, the point cloud data will be processed by the plane extraction method, the nine-square field method, and the image field method in turn, and the point cloud of the drivable area in the point cloud data will be output, and the point cloud of the drivable area will be mapped to the raster map, and the raster map of the drivable area will be output. The autonomous driving car will drive according to the raster map.

2. The method for detecting a driving area of ​​an autonomous driving vehicle according to claim 1, characterized in that: In step 1), the collected point cloud data and image data need to be preprocessed, wherein the point cloud data is subjected to denoising and filtering, and the image data is subjected to distortion correction, color normalization, and denoising.

3. The method for detecting a driving area of ​​an autonomous driving vehicle according to claim 1, characterized in that: In step 1), the timestamps of the laser radar and the surround view camera are based on the same time reference.

4. The method for detecting a driving area of ​​an autonomous driving vehicle according to claim 1, characterized in that: The plane extraction method extracts planes from the initial drivable point cloud using a RANSAC algorithm.

5. The method for detecting a driving area of ​​an autonomous driving vehicle according to claim 1, characterized in that: The image domain method converts the point cloud coordinates into image coordinates through a coordinate conversion formula, and the coordinate conversion formula is: Where [w,h] is the row and column of the image coordinates, [x,y,z] is the point cloud coordinates, θg is the horizontal and vertical resolution of the lidar, and floor is rounded up.

Images