Obstacle detection method based on image semantic segmentation

Abstract

The invention discloses an obstacle detection method based on image semantic segmentation. The method comprises the following steps: 1) acquiring an image in a driving environment of an automatic driving vehicle through a perception camera; 2) the road panorama semantic model performs semantic segmentation on the acquired image information to obtain a grayscale image, the grayscale image comprises a plurality of obstacles of different pixel categories, and the obstacles are provided with different numbers; 3) respectively acquiring a binary image of each obstacle according to the grayscale image, and extracting the contour of the obstacle in each binary image to obtain rough edge pixel points of the obstacle in each binary image; (4) filtering rough edge pixel points of the obstacles in each binary image to obtain final edge pixel points, and (5) calibrating and unifying internal and external parameters of the sensing camera, performing radiation transformation on the final edge pixel points of the obstacles according to the internal and external parameters of the sensing camera, and projecting the final edge pixel points of the obstacles to a BEV space.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving vehicles, and in particular to an obstacle detection method based on image semantic segmentation. Background Art

[0002] With the widespread application of autonomous driving technology, people have higher and higher requirements for the safety of autonomous driving technology. The perception module is a crucial part of autonomous driving technology. The perception module is equivalent to human eyes, which can always perceive the complex environmental information around it. Combined with the map module, planning module, and control module, it enables autonomous driving vehicles to drive safely in complex road environments. The detection of obstacles in the surrounding environment of the vehicle body by the perception module provides important data support for vehicle path planning and control. For example, the perception module can detect the boundaries on both sides of the road to ensure that the autonomous driving vehicle will not cross the boundaries on both sides of the road and drive onto the sidewalk; the perception module can detect other vehicles around it to ensure that it will not collide with other vehicles, etc. Therefore, for the perception module, the research on obstacle detection algorithms is a top priority.

[0003] At present, the perception module of autonomous driving technology mainly perceives and detects obstacles through two methods: radar point cloud information or visual image information. For example, the traditional obstacle detection method mainly extracts feature information through image processing technology and then classifies it. The features used are artificially pre-selected features, such as gradient direction histogram features and size-invariant feature transformation characteristics. However, in the calculation process of this method, the feature extraction of obstacles and the judgment of obstacle types are performed separately, and the regulations on the feature selection of obstacles are also relatively strict, resulting in the slow obstacle detection speed and low detection accuracy of autonomous driving cars using this method when facing more complex scenes. Alternatively, obstacles can be detected using a target detection method based on deep learning, such as the two-stage Faster RCNN series algorithm, the one-stage YOLO series algorithm, etc., which uses a convolutional neural network to extract features and regress the target's location information and category information to obtain the target's center point coordinates, anchor box size and other information. Although this method can quickly and accurately detect obstacles in more complex scenes, it can only obtain two-dimensional data of obstacles without distance information. The perception information in the pixel space cannot be directly used by the downstream planning and control module. At the same time, the output data cannot accurately describe the edge points of the obstacle contour and cannot provide accurate location information for the downstream path planning and control module. Alternatively, obstacles can be detected and identified using a method for segmenting three-dimensional point cloud data. This method uses a laser radar sensor to scan the surrounding scene to obtain three-dimensional point cloud data, and segment the scanned point cloud data to obtain the scale information and category information of the obstacle. This method is mainly divided into feature-based, ground projection-based and depth-based.Figure 3 Ways to achieve: 1. The feature-based method directly constructs complex features in three-dimensional space, and then segments the target through a clustering algorithm. Although this type of feature-based method can achieve good segmentation effects, it is computationally intensive and time-consuming.

[0004] 2. The ground projection method projects the point cloud to the top-down plane and processes it using a grid map. The maximum height difference of the measured values ​​in the cell is used to determine whether the cell is occupied. Obstacles of various shapes are detected by clustering and feature extraction of the occupied cells. This method is simple and efficient, but the grid cell parameters cannot be adaptively adjusted, which can easily cause under-segmentation or over-segmentation.

[0005] 3. The depth map-based method rotates the lidar once to generate multiple scan lines, and uses the scan lines to construct a two-dimensional depth map structure. The method is simple and can use existing mature image processing algorithms to quickly process the depth map. However, the hardware cost is high, which is not conducive to marketization, and the semantic information is relatively lacking compared to images. Summary of the invention

[0006] In response to the above-mentioned problems, the present invention provides an obstacle detection method based on image semantic segmentation, which can quickly and accurately obtain the position of obstacles, provide important data support for path planning and control of unmanned vehicles, and greatly improve the safety of unmanned vehicles.

[0007] The technical solution of the present invention is: an obstacle detection method based on image semantic segmentation, comprising the following steps: 1) Obtain images of the current driving environment of the autonomous vehicle through the perception camera on the car; 2) semantically segmenting the collected image information according to the road panoramic semantic model to obtain a grayscale image, wherein the grayscale image includes obstacles of multiple different pixel categories, and each obstacle is assigned a different number; 3) Obtain the binary images of each obstacle according to the grayscale image, and extract the outline of the obstacle in each binary image to obtain the rough edge pixel points of the obstacle in each binary image; 4) Filter the rough edge pixels of obstacles in each binary image to obtain the final edge pixels; 5) The internal and external parameters of the perception camera are calibrated uniformly, and the final edge pixel points of each obstacle are radially transformed according to the internal and external parameters of the perception camera, and the final edge pixel points of each obstacle are projected into the BEV space.

[0008] Preferably, in step 2), the obstacles include road surfaces, cars, pedestrians, fixed obstacles, and road boundaries.

[0009] Preferably, the road surface is numbered 1, the cars are numbered 2, the pedestrians are numbered 3, the fixed obstacles are numbered 4, and the road boundary line is numbered 5.

[0010] Preferably, in step 3), the opencv function library is called to extract the contours of obstacles in each binary image.

[0011] Preferably, in step 4), the rough edge pixel points of the obstacles in each binary image are traversed by using a four-neighborhood method, and the rough edge pixel points of the obstacles in each binary image are filtered.

[0012] Preferably, in step 5), the internal and external parameters of the perception camera are calibrated uniformly according to the Zhang Zhengyou calibration method.

[0013] The advantages of the present invention are as follows: the present invention segments the image of the body environment of the autonomous driving vehicle obtained by the perception camera through the road panoramic semantic model, extracts the boundary contour points of the obstacles in the image respectively, and then confirms the final boundary contour points of the obstacles in the image by filtering the boundary contour points, and maps the final boundary contour points of each obstacle into the BEV space, so that the autonomous driving vehicle can confirm the exact position of each obstacle, and provides accurate information for the path planning and control of the autonomous driving vehicle, ensuring that the autonomous driving vehicle can avoid obstacles, and also ensuring that the autonomous driving vehicle will not drive out of the road boundary, greatly improving the safety of the autonomous driving vehicle. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 is a flow chart of the present invention; Figure 2 The road surface binarization image of the present invention; Figure 3 A rough edge pixel map of the road surface of the present invention; Figure 4 This is the final edge pixel map of the road surface of the present invention. DETAILED DESCRIPTION

[0015] See also Figure 1 , an obstacle detection method based on image semantic segmentation, comprising the following steps: 1) Obtain images of the current driving environment of the autonomous vehicle through the perception camera on the car; 2) semantically segmenting the collected image information according to the road panoramic semantic model to obtain a grayscale image, wherein the grayscale image includes obstacles of multiple different pixel categories, and each obstacle is provided with a different number; the obstacles include road surfaces, cars, pedestrians, fixed obstacles, and road boundaries, wherein the road surface is numbered 1, the car is numbered 2, the pedestrian is numbered 3, the fixed obstacle is numbered 4, and the road boundary is numbered 5. In the present invention, fixed obstacles refer to buildings on both sides of the road, roadblocks on the road, signboards, etc.

[0016] 3) Obtain the binary images of each obstacle according to the grayscale image, and call the opencv function library to extract the outline of the obstacle in each binary image to obtain the rough edge pixel points of the obstacle in each binary image; in this embodiment, the road surface is taken as an example, see Figure 2 , according to the grayscale image in step 2), obtain the binary image of the road surface, Figure 2 In the figure, the white part represents the road surface and the black part represents other parts. Figure 3 , call the opencv function library to extract the contour of the road surface in the binary image of the road surface and obtain the rough edge pixel points of the road surface.

[0017] 4) Use the four-neighborhood method to traverse the rough edge pixel points of the obstacles in each binary image, filter the rough edge pixel points of the obstacles in each binary image, and obtain the final edge pixel points; see Figure 4 , the rough edge pixel points of the road surface are filtered according to the four-neighborhood method to obtain the final edge pixel point. The four-neighborhood method refers to selecting a rough edge pixel point, taking the rough edge pixel point as the center, confirming whether the numbers of the four points above, below, left, and right of the rough edge pixel point are all the same as the number of the rough edge pixel point. If all are the same, the rough edge pixel point is not the final edge pixel point, and filtering the rough edge pixel point. If the numbers of the four points above, below, left, and right of the rough edge pixel point have at least one point that is the same as the number of the rough edge pixel point If the numbers are different, the coarse edge pixel point is the final edge pixel point. In this embodiment, the road surface is numbered 1. When the four-neighborhood method is used to traverse the coarse edge pixel points of all road surfaces, if the numbers of the upper, lower, left and right points of a coarse edge pixel point are all 1, it means that the coarse edge pixel point is not the final edge pixel point of the road surface and needs to be filtered and discarded. If the numbers of the upper, lower, left and right points of a coarse edge pixel point are not all 1, it means that the coarse edge pixel point is the final edge pixel point of the road surface and the coarse edge pixel point is retained.

[0018] 5) The internal and external parameters of the perception camera are calibrated uniformly according to the Zhang Zhengyou calibration method, and the final edge pixel points of each obstacle are radially transformed according to the internal and external parameters of the perception camera, and the final edge pixel points of each obstacle are projected into the BEV space. The autonomous driving vehicle can determine the exact location of each obstacle based on the image of the BEV space, provide accurate information for the path planning and control of the autonomous driving vehicle, ensure that the autonomous driving vehicle can avoid obstacles, and also ensure that the autonomous driving vehicle will not drive out of the road boundary, greatly improving the safety of the autonomous driving vehicle.

[0019] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modification made to the present invention by those skilled in the art without departing from the spirit of the present invention shall fall within the protection scope of the present invention.

Claims

1. An obstacle detection method based on image semantic segmentation, characterized in that: The following steps are involved: 1) Obtain images of the current driving environment of the autonomous vehicle through the perception camera on the car; 2) semantically segmenting the collected image information according to the road panoramic semantic model to obtain a grayscale image, wherein the grayscale image includes obstacles of multiple different pixel categories, and each obstacle is assigned a different number; 3) Obtain the binary images of each obstacle according to the grayscale image, and extract the outline of the obstacle in each binary image to obtain the rough edge pixel points of the obstacle in each binary image; 4) Filter the rough edge pixels of obstacles in each binary image to obtain the final edge pixels; 5) The internal and external parameters of the perception camera are calibrated uniformly, and the final edge pixel points of each obstacle are radially transformed according to the internal and external parameters of the perception camera, and the final edge pixel points of each obstacle are projected into the BEV space.

2. The obstacle detection method based on image semantic segmentation according to claim 1, characterized in that: In step 2), the obstacles include road surfaces, cars, pedestrians, fixed obstacles, and road boundaries.

3. The obstacle detection method based on image semantic segmentation according to claim 2, characterized in that: The road surface is numbered 1, the car is numbered 2, the pedestrian is numbered 3, the fixed obstacle is numbered 4, and the road boundary line is numbered 5.

4. The obstacle detection method based on image semantic segmentation according to claim 1, characterized in that: In step 3), the opencv function library is called to extract the outlines of obstacles in each binary image.

5. The obstacle detection method based on image semantic segmentation according to claim 1, characterized in that: In step 4), the rough edge pixel points of the obstacles in each binary image are traversed by using the four-neighborhood method, and the rough edge pixel points of the obstacles in each binary image are filtered.

6. The obstacle detection method based on image semantic segmentation according to claim 1, characterized in that: In step 5), the internal and external parameters of the perception camera are calibrated uniformly according to Zhang Zhengyou's calibration method.

Images