A method for detecting drivable area based on semantic segmentation

Abstract

This invention relates to a panoramic drivable area detection method based on semantic segmentation, comprising the following steps: 1) A panoramic camera group acquires images of the vehicle in multiple horizontal directions; 2) Semantic segmentation is performed on the images to obtain grayscale images; 3) The boundaries of drivable areas in the grayscale images are extracted; 4) The boundaries of the drivable areas are transformed from the coordinate system of the grayscale images to the vehicle coordinate system to obtain boundary coordinates; 5) All boundary coordinates are merged to obtain global boundary coordinates; 6) Angle filtering is applied to the global boundary coordinates to obtain the drivable area boundaries. This invention uses a panoramic camera group to acquire images, resulting in a more comprehensive detection of drivable areas; semantic segmentation-based drivable area boundary extraction is accurate and easy; transforming the drivable area boundaries to the vehicle coordinate system reduces the difficulty of fusion; and angle filtering improves the reliability of the detection results, effectively enhancing the safety of autonomous driving.

Description

Technical Field

[0001] The present invention belongs to the technical field of automatic driving of automobiles, and in particular relates to a method for detecting a peripheral visible drivable area based on semantic segmentation. Background Art

[0002] With the rapid development of automobile intelligence, more and more cars are equipped with autonomous driving systems. Detecting and identifying the drivable area from the road image around the vehicle is the basis for the autonomous driving system to achieve autonomous driving control.

[0003] At present, the methods for detecting the drivable area of ​​a road include: monocular vision method, stereoscopic vision method, lidar method and method based on multi-information fusion, etc. Among them, the monocular vision method only considers the environmental information in a single direction, and the detection results are limited, such as when changing lanes or turning around; the stereoscopic vision method takes a huge amount of time in three-dimensional reconstruction and is not suitable for practical application; the lidar method has the disadvantages of sparse point cloud data and difficult detection and recognition; methods based on multi-information fusion, such as Chinese patent CN201710283453.3, a method for detecting the drivable area of ​​a road based on the fusion of monocular vision and lidar, have high cost of lidar and high difficulty in fusion technology. Due to the monocular vision fusion, there is also a problem of large limitations in the detection results.

[0004] Therefore, it is necessary to propose a simple and reliable method that can detect and identify the drivable area of ​​the surrounding environmental information, so as to solve the problems of high technical difficulty and large limitations of the current methods for detecting the drivable area of ​​the road. Summary of the invention

[0005] In view of the above-mentioned deficiencies in the prior art, the technical problem to be solved by the present invention is to provide a peripheral vision drivable area detection method based on semantic segmentation, so as to solve the problems of high detection difficulty and large limitations of detection results in the current drivable area detection methods, and achieve the effect of improving the comprehensiveness of information acquisition and control safety of autonomous driving.

[0006] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0007] A method for detecting a drivable area based on semantic segmentation comprises the following steps:

[0008] 1) The panoramic camera group collects images of the vehicle in multiple horizontal directions;

[0009] 2) Perform semantic segmentation on the collected image to obtain a grayscale image;

[0010] 3) Extract the boundary of the drivable area in the grayscale image;

[0011] 4) Convert the boundary of the drivable area from the grayscale image coordinate system to the vehicle coordinate system to obtain the boundary coordinates;

[0012] 5) Merge the boundary coordinates obtained based on all images to obtain the global boundary coordinates;

[0013] 6) Perform angle filtering on the global boundary coordinates to obtain the boundary of the drivable area.

[0014] Furthermore, in step 1), the panoramic camera group includes a front view camera, a rear view camera, a left view camera, a right view camera, a left rear camera and a right rear camera, and the resolution of images captured by all cameras is the same.

[0015] Furthermore, in step 2), the grayscale image represents different types of targets with different pixel values; in step 3), the boundary of the drivable area refers to the boundary of a continuous barrier-free road centered on the vehicle.

[0016] Furthermore, in step 3), the specific content of extracting the boundary of the drivable area in the grayscale image is as follows: the pixel points in the grayscale image are judged column by column and from bottom to top, and when the pixel value of the pixel point exceeds the pixel value corresponding to the road, the previous pixel point is used as a boundary point of the drivable area.

[0017] Furthermore, in step 4), the coordinate system of the grayscale image takes the upper left corner of the grayscale image as the origin.

[0018] Furthermore, in step 5), the merging refers to merging the boundary coordinates obtained based on all images into one container so as to perform angle filtering in step 6).

[0019] Furthermore, in step 6), the specific content of the angle filtering of the global boundary coordinates is as follows: taking the origin of the vehicle coordinate system as the center of the circle, the area where the vehicle is located is evenly divided into multiple sector-shaped areas, and the global boundary coordinates retain only the point closest to the center of the circle in each sector-shaped area as the boundary point of the drivable area.

[0020] Furthermore, in step 6), the area where the vehicle is located is evenly divided into 144 sector-shaped areas, each sector-shaped area is 2.5 degrees, and the distance between each point in each sector-shaped area and the center of the circle is calculated and compared. The point closest to the center of the circle in each sector-shaped area is taken as the boundary point of the drivable area and saved in a new container to obtain the boundary of the drivable area.

[0021] Compared with the prior art, the present invention has the following beneficial effects:

[0022] 1. The method for detecting a surround-view drivable area based on semantic segmentation described in the present invention collects images of the vehicle in multiple horizontal directions through a surround-view camera group to ensure 360-degree coverage of environmental information around the vehicle, making the detected drivable area more comprehensive and supporting various driving operations such as lane changing and U-turn, thereby effectively solving the problem that the current drivable area detection results are relatively limited.

[0023] 2. The method for detecting the drivable area based on semantic segmentation in the present invention extracts the boundary of the drivable area in the image based on semantic segmentation. Starting from the pixel information of each target in the image, the detection and recognition is more accurate and the operation is simple. Compared with the recognition and detection of the drivable area based on laser radar, the technology is more mature and less difficult.

[0024] 3. The method for detecting the peripheral visible drivable area based on semantic segmentation described in the present invention can effectively solve the problem of high fusion difficulty by converting the boundaries of the drivable area in multiple images into the vehicle coordinate system through coordinate transformation. At the same time, a large number of coordinate points with disorderly distribution are angle filtered to retain the most reliable coordinate points to form the final drivable area boundary, which can effectively ensure the reliability of the detection and recognition results and improve the safety of autonomous driving. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 The present invention is a flowchart of a method for detecting a peripheral visible drivable area based on semantic segmentation according to an embodiment. DETAILED DESCRIPTION

[0026] The specific implementation modes of the present invention will be further described in detail below in conjunction with the accompanying drawings.

[0027] Example:

[0028] See also Figure 1 A method for detecting a drivable area based on semantic segmentation comprises the following steps:

[0029] 1) The panoramic camera group collects images of the vehicle in multiple horizontal directions;

[0030] To ensure that the environmental information around the vehicle can be covered 360 degrees, in this embodiment, the surrounding camera group includes a front view camera, a rear view camera, a left view camera, a right view camera, a left rear camera and a right rear camera;

[0031] In order to avoid affecting the operation speed due to the difference in resolution of the images captured by the six cameras or the resolution being too high, in this embodiment, the resolution of the images captured by all cameras is the same, and the resolution of the images captured by the cameras is 640×480.

[0032] The method for detecting a surround-view drivable area based on semantic segmentation described in the present invention collects images of the vehicle in multiple horizontal directions through a surround-view camera group, thereby ensuring 360-degree coverage of environmental information around the vehicle, making the detected drivable area more comprehensive, and supporting various driving operations such as lane changing and U-turns, thereby effectively solving the problem that the current drivable area detection results are relatively limited.

[0033] 2) Performing semantic segmentation on the collected image through the trained network model to obtain a grayscale image; the grayscale image represents different types of targets with different pixel values; during implementation, the target types may include vehicles, pedestrians, roads, grass, trees, etc.

[0034] 3) Extracting the boundary of the drivable area in the grayscale image; the boundary of the drivable area refers to the boundary of a continuous barrier-free road centered on the vehicle;

[0035] In this embodiment, the method used to extract the boundary of the drivable area in the grayscale image is as follows: the pixels in the grayscale image are judged column by column and from bottom to top, and when the pixel value of the pixel exceeds the pixel value corresponding to the road, the previous pixel is used as a boundary point of the drivable area; in addition, since the shooting range of the front-view camera includes part of the hood of the vehicle, when performing speech segmentation on the image captured by the front-view camera, the hood part in the image can be preset as a road, or when extracting the boundary of the drivable area, the hood part in the image can be cropped out, and the pixel value judgment starts directly from the position in front of the hood.

[0036] The method for detecting a peripheral drivable area based on semantic segmentation described in the present invention extracts the boundary of the drivable area in the image based on semantic segmentation. Starting from the pixel information of each target in the image, the detection and recognition is relatively accurate and the operation is simple. Compared with the identification and detection of the drivable area based on laser radar, the technology is more mature and less difficult.

[0037] 4) Convert the boundary of the drivable area from the coordinate system of the grayscale image to the vehicle coordinate system to obtain boundary coordinates; in this embodiment, the upper left corner of the grayscale image is taken as the origin of the grayscale image coordinate system.

[0038] 5) The boundary coordinates obtained based on all images are merged to obtain the global boundary coordinates. Since it is not conducive to subsequent unified processing to save the boundary coordinates obtained from each image in 6 containers respectively, the boundary coordinates obtained based on all images are merged, that is, the 6 containers are merged into 1 container, and the boundary coordinates obtained based on all images are saved in one container.

[0039] 6) Perform angle filtering on the global boundary coordinates to obtain the boundary of the drivable area;

[0040] Since there are too many coordinate points and they are randomly distributed, subsequent use is difficult, so filtering is required. In this embodiment, the specific operation of angle filtering of the global boundary coordinates is as follows:

[0041] First, with the origin of the vehicle coordinate system as the center of the circle, the area where the vehicle is located is evenly divided into 144 sector-shaped areas, each sector-shaped area is 2.5 degrees; then, the distance between each point in each sector-shaped area and the center of the circle is calculated and compared with each other, and the point closest to the center of the circle is retained as the boundary point of the drivable area; finally, all the boundary points of the drivable area are saved in a new container, and all the boundary points of the drivable area constitute the boundary of the drivable area.

[0042] The method for detecting a peripheral visible drivable area based on semantic segmentation described in the present invention can effectively solve the problem of high fusion difficulty by converting the boundaries of the drivable area in multiple images into the vehicle coordinate system through coordinate transformation; at the same time, a large number of coordinate points with disorderly distribution are angle filtered, and the most reliable coordinate points are retained to form the final drivable area boundary, which can effectively ensure the reliability of the detection and recognition results and improve the safety of autonomous driving.

[0043] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution of the present invention, which should be included in the scope of the claims of the present invention.

Claims

1. A method for detecting a drivable area based on semantic segmentation, characterized in that: The steps include: 1) The panoramic camera group collects images of the vehicle in multiple horizontal directions; 2) Perform semantic segmentation on the collected image to obtain a grayscale image; 3) Extract the boundary of the drivable area in the grayscale image; 4) Convert the boundary of the drivable area from the grayscale image coordinate system to the vehicle coordinate system to obtain the boundary coordinates; 5) Merge the boundary coordinates obtained based on all images to obtain the global boundary coordinates; 6) Perform angle filtering on the global boundary coordinates to obtain the boundary of the drivable area; In step 2), the grayscale image represents different types of targets with different pixel values; in step 3), the boundary of the drivable area refers to the boundary of a continuous barrier-free road centered on the vehicle; In step 3), the specific content of extracting the boundary of the drivable area in the grayscale image is as follows: judging the pixel points in the grayscale image column by column and from bottom to top, when the pixel value of the pixel point exceeds the pixel value corresponding to the road, the previous pixel point is used as a boundary point of the drivable area; In step 6), the specific content of the angle filtering of the global boundary coordinates is as follows: with the origin of the vehicle coordinate system as the center of the circle, the area where the vehicle is located is evenly divided into multiple sector-shaped areas, and the global boundary coordinates retain only the point closest to the center of the circle in each sector-shaped area as the boundary point of the drivable area.

2. The method for detecting a peripheral visible drivable area based on semantic segmentation according to claim 1, characterized in that: In step 1), the panoramic camera group includes a front view camera, a rear view camera, a left view camera, a right view camera, a left rear camera and a right rear camera, and the resolution of images captured by all cameras is the same.

3. The method for detecting a peripheral visible drivable area based on semantic segmentation according to claim 1, characterized in that: In step 4), the coordinate system of the grayscale image takes the upper left corner of the grayscale image as the origin.

4. The method for detecting a peripheral visible drivable area based on semantic segmentation according to claim 1, characterized in that: In step 5), the merging refers to merging the boundary coordinates obtained based on all images into one container so as to perform angle filtering in step 6).

5. The method for detecting a peripheral visible drivable area based on semantic segmentation according to claim 1, characterized in that: In step 6), the area where the vehicle is located is evenly divided into 144 sectors, each sector is 2.5 degrees, the distance between each point in each sector and the center of the circle is calculated and compared, and the point closest to the center of the circle in each sector is taken as the boundary point of the drivable area and saved in a new container to obtain the boundary of the drivable area.

Images