Vehicle drivable area detection method, related device and storage medium

Abstract

The present application relates to the field of computer vision technology, and provides a vehicle drivable area detection method, related device and storage medium, the method comprising: obtaining target point cloud data, which includes first point cloud data of the current frame and second point cloud data of the historical frame within a preset time window; accumulating the first point cloud data and the second point cloud data to fill the first point cloud data to obtain third point cloud data; clustering the filled third point cloud data to obtain at least one obstacle in the current frame; performing raster binarization on the contours of each obstacle extracted to obtain a candidate raster map, removing the non-drivable area from the candidate raster map, obtaining the effective drivable area of ​​the target vehicle in the current frame and outputting it. This solution can improve the detection accuracy and effectiveness of automatic parking, reduce the deviation of the detection result from the actual drivable area, and avoid the situation where the actual non-drivable area is misjudged as a drivable area.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of computer vision technology, and in particular to a vehicle drivable area detection method, related devices, and storage media. Background Art

[0002] In the automatic parking scenario, high-precision drivable area detection is required. The current mainstream detection methods include: ultrasonic detection, vision or ultrasonic radar fusion detection, radar detection (such as millimeter wave radar, lidar detection), etc.

[0003] For the detection method based on vision or ultrasonic radar fusion, this detection method uses a deep learning algorithm, which usually requires manual annotation of obstacles in the training data, and training the features of the drivable and non-drivable areas based on the manually annotated training data, and then separating the road surface and obstacles based on feature information such as color, texture, and edge. Although the accuracy is high, due to the long detection distance of this detection method, a large amount of manual sample collection is required in the early stage, and the reasoning process requires high chip computing power, and real-time performance is difficult to guarantee. In addition, it has high requirements for ambient lighting, and the scene robustness is limited. Summary of the invention

[0004] The embodiments of the present application provide a vehicle drivable area detection method, related devices and storage media, which can improve the detection accuracy and effectiveness of drivable areas and non-drivable areas in automatic parking scenarios, and effectively reduce the deviation of detection results from the actual drivable areas, so as to avoid the situation where the actual non-drivable areas are misjudged as drivable areas. The use of this solution can effectively avoid the adverse effects on subsequent parking space detection and parking path planning due to the low accuracy of detection results.

[0005] In a first aspect, an embodiment of the present application provides a vehicle drivable area detection method from the perspective of a vehicle-mounted device, the method comprising:

[0006] Acquire target point cloud data, the target point cloud data including first point cloud data of a current frame and second point cloud data of a historical frame within a preset time window, the current frame being an image of a drivable area detected by a radar installed on the target vehicle, and the target point cloud data including a point cloud set detected by the radar;

[0007] Accumulating the first point cloud data and the second point cloud data to fill the first point cloud data to obtain third point cloud data;

[0008] Clustering the filled third point cloud data to obtain at least one obstacle in the current frame, each obstacle covering at least one point cloud;

[0009] Extract the outline of each obstacle;

[0010] Binarize the outline of each obstacle to obtain a candidate grid map, wherein the candidate grid map includes a drivable area and a non-drivable area corresponding to the target vehicle in the current frame;

[0011] The non-drivable area is removed from the candidate grid map to obtain and output a valid drivable area for the target vehicle in the current frame.

[0012] In some implementations, the historical frame includes any first frame and second frame within the preset time window, the timestamp of the first frame is earlier than that of the second frame, and the first frame is the frame with the earliest timestamp in the point cloud accumulation of the second frame;

[0013] The accumulating the first point cloud data and the second point cloud data to fill the first point cloud data to obtain third point cloud data includes:

[0014] Get the real-time speed and frame period of the target vehicle corresponding to the current frame;

[0015] Acquire the point cloud accumulation result of the second frame from the second point cloud data, and obtain the point cloud data of the first frame with the earliest timestamp in the point cloud accumulation result of the second frame;

[0016] According to the real-time vehicle speed and the frame period, motion compensation is performed on the current frame and filled into the first point cloud data of the current frame to obtain the third point cloud data.

[0017] In some implementations, binarizing the grid according to the outlines of each obstacle to obtain a candidate grid map includes:

[0018] Calculate the line equations of the edge point clouds corresponding to the outlines of each obstacle. Each line L between the edge point clouds i Expressed as slope k i , intercept b i , minimum vertical coordinate y imin , maximum longitudinal coordinate y imax combination of;

[0019] According to the grid granularity, traverse the longitudinal coordinate y of each grid center point n , calculate the vertical coordinate as y n When each line L i The horizontal coordinate of the intersection point is x ni , for each grid center point longitudinal coordinate y n Assign multiple intersection points lateral coordinates x nj , where j = 1, 2, ...;

[0020] According to the grid granularity, traverse the longitudinal coordinate y of each grid center point n All grids under the grid are scanned from left to right. If the center point of the leftmost grid is outside the outline area of ​​the obstacle, a value of 1 is assigned. If the center point of the leftmost grid is inside the outline area of ​​the obstacle, a value of 0 is assigned. Scan the grids to the right and repeat the assignment. During the scanning process, the horizontal coordinate x of each intersection is nj , then a negation operation is performed until all grids are assigned values ​​to obtain the drivable area and the non-drivable area.

[0021] In some implementations, removing the non-drivable area from the candidate grid map to obtain a valid drivable area for the target vehicle in the current frame includes:

[0022] Scan each row of grids G in the candidate grid map in sequence yi , find all connected first continuous grids;

[0023] Scan each column grid G ​​in the candidate grid map in turn xi , find the connected second continuous grid;

[0024] The effective drivable area is obtained according to the first continuous grid and the second continuous grid.

[0025] In some implementations, after clustering the filled third point cloud data and before extracting the contours of each obstacle, the method further includes:

[0026] From the third point cloud, point clouds with heights between the chassis of the target vehicle and the total height of the target vehicle are selected, and invalid point clouds are deleted to obtain valid point clouds constituting actual obstacles.

[0027] In some implementations, the point cloud coordinates of the third point cloud or the valid point cloud projected on the ground are set to P i (x i ,y i ); after clustering the filled third point cloud data and before extracting the contours of each obstacle, the method further includes:

[0028] Set the safety distance δ;

[0029] Based on the safety distance, the third point cloud or the valid point cloud is plane-expanded so that each point cloud coordinate P projected on the ground i (x i ,y i ) are replaced by a target point cloud, wherein the target point cloud includes at least one of the following items:

[0030] P i1(x i +δ,y i +δ), P i2 (x i -δ,y i +δ), P i3 (x i -δ,y i -δ), and P i4 (x i +δ,y i -δ).

[0031] In some implementations, before acquiring the point cloud data of the target vehicle, the method further includes:

[0032] Determining the detection range and grid granularity of the radar;

[0033] A grid map of the target vehicle is established according to the detection distance and the grid granularity, and the detection range of the radar covers the grid map.

[0034] In a second aspect, an embodiment of the present application provides a vehicle-mounted device having a function of implementing the vehicle drivable area detection method provided in the first aspect above. The function can be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions, and the modules can be software and / or hardware.

[0035] In some implementations, the vehicle-mounted device includes:

[0036] An input-output module, used to obtain target point cloud data, the target point cloud data including first point cloud data of a current frame and second point cloud data of a historical frame within a preset time window, the current frame being an image of a drivable area detected by a radar installed on the target vehicle, and the target point cloud data including a point cloud set detected by the radar;

[0037] A processing module, configured to accumulate the first point cloud data and the second point cloud data to fill the first point cloud data to obtain third point cloud data;

[0038] Clustering the filled third point cloud data to obtain at least one obstacle in the current frame, each obstacle covering at least one point cloud;

[0039] Extract the outline of each obstacle;

[0040] Binarize the outline of each obstacle to obtain a candidate grid map, wherein the candidate grid map includes a drivable area and a non-drivable area corresponding to the target vehicle in the current frame;

[0041] The non-drivable area is removed from the candidate grid map to obtain a valid drivable area of ​​the target vehicle in the current frame and output it through the input-output module.

[0042] In some implementations, the historical frame includes any first frame and second frame within the preset time window, the timestamp of the first frame is earlier than that of the second frame, and the first frame is the frame with the earliest timestamp in the point cloud accumulation of the second frame;

[0043] The processing module is specifically used for:

[0044] Get the real-time speed and frame period of the target vehicle corresponding to the current frame;

[0045] Acquire the point cloud accumulation result of the second frame from the second point cloud data, and obtain the point cloud data of the first frame with the earliest timestamp in the point cloud accumulation result of the second frame;

[0046] According to the real-time vehicle speed and the frame period, motion compensation is performed on the current frame and filled into the first point cloud data of the current frame to obtain the third point cloud data.

[0047] In some implementations, the processing module is specifically used to:

[0048] Calculate the line equations of the edge point clouds corresponding to the outlines of each obstacle. Each line L between the edge point clouds i Expressed as slope k i , intercept b i , minimum vertical coordinate y imin , maximum longitudinal coordinate y imax combination of;

[0049] According to the grid granularity, traverse the longitudinal coordinate y of each grid center point n , calculate the vertical coordinate as y n When each line L i The horizontal coordinate of the intersection point is x ni , for each grid center point longitudinal coordinate y n Assign multiple intersection points lateral coordinates x nj , where j = 1, 2, ...;

[0050] According to the grid granularity, traverse the longitudinal coordinate y of each grid center point n All grids under the grid are scanned from left to right. If the center point of the leftmost grid is outside the outline area of ​​the obstacle, a value of 1 is assigned. If the center point of the leftmost grid is inside the outline area of ​​the obstacle, a value of 0 is assigned. Scan the grids to the right and repeat the assignment. During the scanning process, the horizontal coordinate x of each intersection is nj, then a negation operation is performed until all grids are assigned values ​​to obtain the drivable area and the non-drivable area.

[0051] In some implementations, the processing module is specifically used to:

[0052] Scan each row of grids G in the candidate grid map in sequence yi , find all connected first continuous grids;

[0053] Scan each column grid G ​​in the candidate grid map in turn xi , find the connected second continuous grid;

[0054] The effective drivable area is obtained according to the first continuous grid and the second continuous grid.

[0055] In some implementations, after clustering the filled third point cloud data and before extracting the contours of each obstacle, the processing module is further configured to:

[0056] From the third point cloud, point clouds with heights between the chassis of the target vehicle and the total height of the target vehicle are selected, and invalid point clouds are deleted to obtain valid point clouds constituting actual obstacles.

[0057] In some implementations, the point cloud coordinates of the third point cloud or the valid point cloud projected on the ground are set to P i (x i ,y i ); after clustering the filled third point cloud data and before extracting the contours of each obstacle, the method further includes:

[0058] Set the safety distance δ;

[0059] Based on the safety distance, the third point cloud or the valid point cloud is plane-expanded so that each point cloud coordinate P projected on the ground i (x i ,y i ) are replaced by a target point cloud, wherein the target point cloud includes at least one of the following items:

[0060] P i1 (x i +δ,y i +δ), P i2 (x i -δ,y i +δ), P i3 (x i -δ,y i -δ), and P i4 (x i +δ,y i-δ).

[0061] In some implementations, before acquiring the point cloud data of the target vehicle, the processing module is further used to:

[0062] Determining the detection range and grid granularity of the radar;

[0063] A grid map of the target vehicle is established according to the detection distance and the grid granularity, and the detection range of the radar covers the grid map.

[0064] In the third aspect, an embodiment of the present application provides a vehicle-mounted face device, which includes: at least one processor and a memory; wherein the memory is used to store a computer program, and the processor is used to call the computer program stored in the memory to execute the steps in the above-mentioned first aspect and any one of the implementations of the first aspect.

[0065] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a function of implementing the vehicle drivable area detection method provided in the first aspect above. The function can be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions, and the modules can be software and / or hardware. Specifically, the computer-readable storage medium stores a plurality of instructions, and the instructions are suitable for a processor to load to execute the steps in the first aspect and any one of the implementations of the first aspect in the embodiment of the present application.

[0066] Compared with the prior art, in the solution provided by the embodiment of the present application, the target point cloud data is first obtained, which includes the first point cloud data of the current frame and the second point cloud data of the historical frame within the preset time window; the first point cloud data and the second point cloud data are accumulated to fill the first point cloud data to obtain the third point cloud data; the filled third point cloud data is clustered to obtain at least one obstacle in the current frame; the contours of each obstacle extracted are rasterized to obtain a candidate raster map, and the non-drivable area is removed from the candidate raster map to obtain the effective drivable area of ​​the target vehicle in the current frame and output. Since the edge detection adopts the target-level detection of the accumulated point cloud instead of the point cloud-level detection, the accuracy is higher, and the detection accuracy and effectiveness of the drivable area and the non-drivable area in the automatic parking scenario can be improved, and the deviation of the detection result from the actual drivable area can be effectively reduced to avoid the situation where the actual non-drivable area is misjudged as a drivable area. The use of this solution can effectively avoid the adverse effects on the later parking space detection and parking path planning caused by the low accuracy of the detection result. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] Figure 1A schematic diagram of a flow chart of a method for detecting a vehicle drivable area in an embodiment of the present application;

[0068] Figure 2a This is a schematic diagram of the installation and blind area (gray area) of five 4D millimeter wave radars in an embodiment of the present application;

[0069] Figure 2b A schematic diagram of a process of point cloud expansion in an embodiment of the present application;

[0070] Figure 3 A schematic diagram of an algorithm flow of a vehicle drivable area detection method in an embodiment of the present application;

[0071] Figure 4a This is a real scene picture of the current frame in the embodiment of the present application (the upper one is the foreground, and the lower one is the background);

[0072] Figure 4b It is a projection diagram of the accumulated point cloud of the current frame (invalid point cloud has been removed) in the embodiment of the present application;

[0073] Figure 4c is a grid map of the drivable area obtained by a traditional algorithm in an embodiment of the present application;

[0074] Figure 4d In the embodiment of this application, Figure 3 The drivable area grid map obtained by the algorithm process shown;

[0075] Figure 5 It is a structural schematic diagram of a vehicle-mounted device for implementing a vehicle drivable area detection method in an embodiment of the present application;

[0076] Figure 6 It is a structural schematic diagram of a physical device for implementing the vehicle drivable area detection method in an embodiment of the present application. DETAILED DESCRIPTION

[0077] The terms "first", "second", etc. in the description and claims of the embodiments of the present application and the above drawings are used to distinguish similar objects (for example, the first point cloud data and the second point cloud data in the embodiments of the present application respectively represent different point cloud data with the same attributes), and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or modules is not necessarily limited to those steps or modules that are clearly listed, but may include other steps or modules that are not clearly listed or inherent to these processes, methods, products or devices. The division of modules that appear in the embodiments of the present application is only a logical division. There may be other division methods when implemented in actual applications. For example, multiple modules can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between each other shown or discussed can be through some interfaces, and the indirect coupling or communication connection between modules can be electrical or other similar forms, which are not limited in the embodiments of the present application. In addition, the modules or submodules described as separate components may or may not be physically separated, may or may not be physical modules, or may be distributed in multiple circuit modules, and some or all of the modules can be selected according to actual needs to achieve the purpose of the embodiment of the present application.

[0078] The embodiments of the present application provide a vehicle drivable area detection method, related devices and storage media, which can be used in a server or terminal device. Specifically, it can be used in an automatic parking scenario to perform high-precision and effective detection of drivable areas and non-drivable areas, effectively reducing the deviation of the detection result from the actual drivable area, so as to avoid the situation where the actual non-drivable area is misjudged as a drivable area. The use of this solution can effectively avoid the adverse effects on subsequent parking space detection and parking path planning due to the low accuracy of the detection results.

[0079] The solution provided in the embodiment of the present application involves technologies such as artificial intelligence (AI), natural language processing (NLP), and machine learning (ML).

[0080] Among them, it should be particularly noted that the server involved in the embodiments of the present application (such as a business server, a search engine) can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The vehicle-mounted device involved in the embodiments of the present application can be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, a personal digital assistant, etc., but is not limited to this. The vehicle-mounted device and the server can be directly or indirectly connected via wired or wireless communication, and the embodiments of the present application are not limited here.

[0081] For the related technology based on vision or ultrasonic radar fusion detection, this detection method uses a deep learning algorithm, which usually requires manual annotation of obstacles in the training data, and training the features of the drivable area and the non-drivable area based on the manually annotated training data, and then separating the road surface and obstacles based on color, texture, edge and other feature information. Although the accuracy is high, due to the long detection distance of this detection method, a large amount of manual sample collection is required in the early stage, and the reasoning process has high requirements on chip computing power, and it is difficult to ensure real-time performance. In addition, it has high requirements on ambient lighting, and the scene robustness is limited.

[0082] In this regard, the embodiment of the present application improves the algorithm based on the traditional 4D millimeter wave radar point cloud detection solution, improves the accuracy of drivable area detection, and mainly adopts the following technical solutions:

[0083] 1. Edge detection uses object-level detection of accumulated point clouds instead of point cloud-level detection, which is more accurate.

[0084] 2. Expand the point cloud that forms the target, then perform convex hull detection to obtain the edge, and use this edge as the boundary of the drivable area. Point cloud expansion can offset the measurement error and the estimation error of the drivable area boundary caused by rasterization, especially to prevent the situation where the boundary of the non-drivable area is detected as a drivable area. Convex hull detection replaces the traditional rectangular box detection, and the description of the boundary contour is more detailed and accurate.

[0085] 3. Adding detection of the reachability of the drivable area. The present invention removes the drivability of areas that the vehicle cannot actually reach, such as narrow gaps and small spaces between obstacles, to prevent them from having an adverse effect on subsequent parking space detection and parking path planning.

[0086] The following combination Figure 1-Figure 6 The technical solution of the embodiment of the present application is exemplified.

[0087] In some implementations, before performing drivable area detection, the vehicle-mounted device needs to pre-build a grid map, as follows: determine the detection distance and grid granularity of the radar, establish a grid map of the target vehicle based on the detection distance and the grid granularity, and the detection range of the radar covers the grid map. For example, the grid map is centered on the vehicle, and the map size depends on the detection distance of the 4D millimeter-wave radar and the actual application requirements. For parking scenarios, the typical map size is 25×50m. The grid granularity depends on the accuracy requirements of the later decision-making algorithm, usually 5 to 20cm.

[0088] like Figure 1 As shown, Figure 1 This is a flow chart of a method for detecting a vehicle drivable area in an embodiment of the present application. The method can be used for a vehicle-mounted device (also referred to as a vehicle-mounted terminal, not limited to this) loaded on a target vehicle. The method includes:

[0089] 101. Obtain target point cloud data.

[0090] Among them, the target point cloud data includes first point cloud data of the current frame and second point cloud data of the historical frame within a preset time window. The current frame is an image of the drivable area detected by a radar installed on the target vehicle, and the target point cloud data includes a point cloud set detected by the radar.

[0091] Specifically, the coordinates of the entire point cloud of the current frame are input. The entire point cloud refers to the point cloud set P detected by the entire 4D millimeter wave radar installed on the vehicle body. i The coordinates are the three-dimensional Euclidean coordinates (x i ,y i ,z i ), and the ground projection point of the rear axle center of the target vehicle is taken as the coordinate origin, the horizontal direction is the x-axis direction, the vertical direction is the y-axis direction, and the height direction is the z-axis direction.

[0092] It should be noted that when the embodiment of the present application performs calculations based on the 4D millimeter wave point cloud, the radar detection range needs to cover the entire grid map, but a small blind spot is allowed outside the radar FOV and in non-decision-making important areas. A typical installation scheme uses a total of 5 radars, including 1 forward radar, 1 left front radar, 1 right front radar, 1 left rear radar, and 1 right rear radar. The radar installation and blind spot diagram is shown in the figure below. Figure 2a shown.

[0093] 102. Accumulate the first point cloud data and the second point cloud data to fill the first point cloud data to obtain third point cloud data.

[0094] In some implementations, the historical frame includes any first frame and second frame within the preset time window, the timestamp of the first frame is earlier than that of the second frame, and the first frame is the frame with the earliest timestamp in the point cloud accumulation of the second frame;

[0095] The accumulating the first point cloud data and the second point cloud data to fill the first point cloud data to obtain third point cloud data includes:

[0096] Get the real-time speed and frame period of the target vehicle corresponding to the current frame;

[0097] Acquire the point cloud accumulation result of the second frame from the second point cloud data, and obtain the point cloud data of the first frame with the earliest timestamp in the point cloud accumulation result of the second frame;

[0098] According to the real-time vehicle speed and the frame period, motion compensation is performed on the current frame and filled into the first point cloud data of the current frame to obtain the third point cloud data.

[0099] For example, a fixed-length sliding window method can be used to accumulate point cloud information of the current frame and the most recent historical frame, totaling N frames, where N is usually 3 to 10. The point cloud accumulation method is to remove the point cloud of the earliest frame from the result of the previous frame point cloud accumulation, then perform motion compensation based on the vehicle speed and frame period, and finally fill in the point cloud information of the current frame.

[0100] 103. Cluster the filled third point cloud data to obtain at least one obstacle in the current frame, each obstacle covering at least one point cloud.

[0101] In some implementations, the DBSCAN algorithm can be used for clustering, where the clustering distance between point clouds is the three-dimensional Euclidean distance, and the threshold is determined according to the density of the current radar output point cloud, usually 0.5-1.0m; the minimum number of clustering points is also adjusted according to the actual effect, usually 5-10. Point cloud clustering classifies point clouds into obstacle targets (clusters), each target contains several point clouds. By adjusting the parameters, each real target is clustered as much as possible, while discrete point clouds (false targets) are not clustered.

[0102] 104. Extract the outline of each obstacle.

[0103] In some implementations, a conventional convex hull detection algorithm can be used to extract the contours of each obstacle, and the calculation process uses the Graham scanning method. After the contour of each target is extracted, only the edge point cloud is retained, and the remaining point clouds are discarded. The connecting lines of the edge point clouds constitute the target contour. It can be seen that when detecting edges, by using the target-level detection of the accumulated point cloud instead of the point cloud-level detection, the accuracy is higher.

[0104] 105. Binarize the outline of each obstacle to obtain a candidate grid map.

[0105] The candidate grid map includes a drivable area and a non-drivable area corresponding to the target vehicle in the current frame.

[0106] In some implementations, binarizing the grid according to the outlines of each obstacle to obtain a candidate grid map includes:

[0107] Calculate the line equations of the edge point clouds corresponding to the outlines of each obstacle. Each line L between the edge point clouds i Expressed as slope k i , intercept b i , minimum vertical coordinate y imin , maximum longitudinal coordinate y imax combination of;

[0108] According to the grid granularity, traverse the longitudinal coordinate y of each grid center point n , calculate the vertical coordinate as y n When each line L i The horizontal coordinate of the intersection point is x ni , for each grid center point longitudinal coordinate y n Assign multiple intersection points lateral coordinates x nj , where j = 1, 2, ...;

[0109] According to the grid granularity, traverse the longitudinal coordinate y of each grid center point n All grids under the grid are scanned from left to right. If the center point of the leftmost grid is outside the outline area of ​​the obstacle, a value of 1 is assigned. If the center point of the leftmost grid is inside the outline area of ​​the obstacle, a value of 0 is assigned. Scan the grids to the right and repeat the assignment. During the scanning process, the horizontal coordinate x of each intersection is nj , then a negation operation is performed until all grids are assigned values ​​to obtain the drivable area and the non-drivable area.

[0110] It can be seen that by adopting the method of horizontal intersection scanning, the grid can be effectively divided into the target outer grid (drivable, set to 1) and the target inner grid (non-drivable, set to 0) according to the overall target contour information, and this method can significantly simplify the algorithm steps, thereby improving the detection efficiency while ensuring the detection accuracy.

[0111] 107. Remove the non-drivable area from the candidate grid map, obtain the valid drivable area of ​​the target vehicle in the current frame, and output it.

[0112] The non-drivable area refers to a grid area in the initial drivable area that is actually impossible to drive into. The most common non-reachable areas are narrow gaps and small spaces between obstacles, which the target vehicle cannot drive into.

[0113] Since judging the non-drivable area as a drivable area will inevitably affect the subsequent parking space detection and parking path planning process, it needs to be eliminated. The ideal inaccessible area detection algorithm is relatively complex. In some implementations, the embodiment of the present application proposes an algorithm for scanning the width of the horizontal and vertical connected areas for implementation. Specifically, removing the non-drivable area from the candidate grid map to obtain the effective drivable area of ​​the target vehicle in the current frame includes:

[0114] (1) Scan each row of grids G in the candidate grid map horizontally yi , find all connected first continuous grids.

[0115] For example, scan each row of grid G ​​in turn yi , find all connected segments, that is, continuous grids set to 1, that is, the first continuous grid. If the segment length is not less than w, it is retained; otherwise, all grids in the segment are set to 0 and regarded as a non-drivable area.

[0116] (2) Scan each column of grids G in the candidate grid map vertically xi , find the connected second continuous grid.

[0117] Scan each column grid G ​​in turn xi , find all connected segments, that is, continuous grids set to 1. If the segment length is not less than w, it is retained; otherwise, all grids in the segment are set to 0 and regarded as a non-drivable area.

[0118] (3) Obtaining the effective drivable area according to the first continuous grid and the second continuous grid.

[0119] After the calculations from steps (1) to (3) above, a grid map of the drivable area in the current frame can be obtained and output.

[0120] Optionally, in some embodiments of the present application, after clustering the filled third point cloud data and before extracting the outline of each obstacle, it is also possible to add a detection of the reachability of the drivable area. Specifically, the method further includes:

[0121] From the third point cloud, point clouds with heights between the chassis of the target vehicle and the total height of the target vehicle are selected, and invalid point clouds are deleted to obtain valid point clouds constituting actual obstacles.

[0122] Specifically, effective point cloud extraction. The effective point cloud refers to the point cloud with height z in all clusters. i The point cloud between the chassis and the total height of the vehicle can retain a certain margin. The valid point cloud constitutes the actual obstacle, and the rest is invalid point cloud and is removed. i No longer used, subsequent steps are based on its ground projection coordinates (x i ,y i )conduct.

[0123] It can be seen that by increasing the reachability detection of the drivable area, it is possible to effectively eliminate the drivability of areas that the vehicle cannot actually reach, such as narrow gaps and small spaces between obstacles, to prevent them from having an adverse effect on subsequent parking space detection and parking path planning.

[0124] Optionally, in some embodiments of the present application, the boundary contour description can be made more detailed and accurate, and a point cloud expansion method can be used. Point cloud expansion refers to the planar expansion of all valid point clouds, the purpose of which is to offset the measurement error and the estimation error of the drivable area boundary caused by rasterization, especially to prevent the situation where the non-drivable area at the boundary is detected as a drivable area. At the same time, point cloud expansion can also provide a set safety distance as a buffer to further prevent the occurrence of scratch accidents.

[0125] Specifically, the point cloud coordinates of the third point cloud or the valid point cloud projected on the ground are set to P i (x i ,y i ); after clustering the filled third point cloud data and before extracting the contours of each obstacle, the method further includes:

[0126] Set the safety distance δ;

[0127] Based on the safety distance, the third point cloud or the valid point cloud is plane-expanded so that each point cloud coordinate P projected on the ground i (x i ,y i ) are replaced by a target point cloud, wherein the target point cloud includes at least one of the following items:

[0128] P i1 (x i +δ,y i +δ), P i2 (x i -δ,y i +δ), P i3 (x i -δ,y i -δ), and P i4 (x i+δ,y i -δ).

[0129] It can be seen that by adding the point cloud expansion method to perform planar expansion on all valid point clouds, the point cloud that forms the target can be expanded, and then the convex hull detection is performed to obtain the edge, and this edge is used as the boundary of the drivable area. Point cloud expansion can offset the measurement error and the estimation error of the drivable area boundary caused by rasterization, especially to prevent the situation where the boundary non-drivable area is detected as a drivable area. Convex hull detection replaces the traditional rectangular frame detection, and the description of the boundary contour is more detailed and accurate.

[0130] In a nutshell, in the embodiment of the present application, the target point cloud data is first obtained, which includes the first point cloud data of the current frame and the second point cloud data of the historical frame within the preset time window; the first point cloud data and the second point cloud data are then accumulated to fill the first point cloud data to obtain the third point cloud data; the filled third point cloud data is clustered to obtain at least one obstacle in the current frame; the contours of each obstacle extracted are rasterized to obtain a candidate raster map, and the non-drivable area is removed from the candidate raster map to obtain the effective drivable area of ​​the target vehicle in the current frame and output. Since the edge detection adopts the target-level detection of the accumulated point cloud instead of the point cloud-level detection, the accuracy is higher, and the detection accuracy and effectiveness of the drivable area and the non-drivable area in the automatic parking scenario can be improved, and the deviation of the detection result from the actual drivable area can be effectively reduced to avoid the situation where the actual non-drivable area is misjudged as a drivable area. The use of this solution can effectively avoid the adverse effects on the later parking space detection and parking path planning due to the low accuracy of the detection result.

[0131] For ease of understanding, the present application embodiment takes the parking scene as an example to illustrate the vehicle drivable area detection method in the present application embodiment. Figure 3-4d . Figure 3 This is an algorithm flow chart of a method for detecting a vehicle drivable area in an embodiment of the present application. Figure 4a is the current frame real scene photo (the upper one is the foreground, the lower one is the background); Figure 4b Cumulative point cloud projection for the current frame (invalid point cloud removed); Figure 4c The grid map of the drivable area obtained by the traditional algorithm; Figure 4d This is a grid map of the drivable area obtained by the algorithm of the present invention.

[0132] The obstacles are cars, pillars and pedestrians. Figure 4a As shown; the cumulative point cloud (invalid point cloud has been removed) is projected as Figure 4b As shown; the drivable area detection results of the traditional algorithm in the industry are as follows Figure 4c As shown; the drivable area detection result of the present invention is as follows Figure 4d As shown (the dark part is the drivable area).

[0133] It can be clearly seen from the test results that the traditional drivable area detection algorithm in the industry is more dependent on point clouds. When the point cloud density is not enough, such as when the target is far away or the reflection is not strong, there are a lot of sparse point clouds in the target, and the sparse point clouds are very likely to be misjudged as drivable areas. Traditional algorithms can alleviate this problem by expanding the radiation range of point clouds, but it is difficult to take into account the misjudgment and over-compression of drivable areas at the same time. On the other hand, there are a large number of fragmented and incoherent drivable areas in the results of traditional algorithms, most of which are inaccessible areas. Sending this result to the subsequent parking space detection and parking path planning algorithms will inevitably have an adverse effect. In comparison, the algorithm of the present invention adopts a target-level edge detection scheme, which can better outline the outline of the obstacle target and delineate the drivable area. At the same time, the algorithm of the present invention effectively eliminates the inaccessible area, making the drivable area more coherent and realistic.

[0134] In summary, the algorithm provided in the embodiment of the present application significantly improves the accuracy of drivable area detection and has better application prospects. Compared with traditional algorithms, the complexity of this algorithm is not much increased. Implementing it in a domain controller chip commonly used in the industry can fully meet the real-time requirements of the system, thereby achieving relatively ideal performance results.

[0135] Figures 1 to 6 Any technical feature mentioned in any embodiment corresponding to any one of the items is also applicable to the embodiments of the present application. Figures 3 to 6 The corresponding embodiments will not be described in detail later.

[0136] A vehicle drivable area detection method in an embodiment of the present application is described above. A vehicle-mounted device for executing the vehicle drivable area detection method is introduced below.

[0137] See also Figure 6 ,like Figure 6 The structure diagram of a vehicle-mounted device 40 shown in FIG. 1 can be applied to an automatic parking scenario to accurately detect the drivable area. The vehicle-mounted device 40 in the embodiment of the present application can achieve the above Figure 1 -Steps in the vehicle drivable area detection method executed by the vehicle-mounted device 40 in any of the embodiments corresponding to FIG. 2. The functions implemented by the vehicle-mounted device 40 can be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions, and the modules can be software and / or hardware. The vehicle-mounted device 40 may include an input / output module 401 and a processing module 402. The functions of the input / output module 401 and the processing module 402 can be implemented by reference to Figure 1-Figure 6The operations performed in any corresponding embodiment are not described in detail here.

[0138] In some embodiments, the input-output module 401 may be used to obtain target point cloud data, the target point cloud data including first point cloud data of a current frame and second point cloud data of a historical frame within a preset time window, the current frame being an image of a drivable area detected by a radar installed on the target vehicle, and the target point cloud data including a point cloud set detected by the radar;

[0139] The processing module 402 is used to accumulate the first point cloud data and the second point cloud data to fill the first point cloud data to obtain third point cloud data;

[0140] Clustering the filled third point cloud data to obtain at least one obstacle in the current frame, each obstacle covering at least one point cloud;

[0141] Extract the outline of each obstacle;

[0142] Binarize the outline of each obstacle to obtain a candidate grid map, wherein the candidate grid map includes a drivable area and a non-drivable area corresponding to the target vehicle in the current frame;

[0143] The non-drivable area is removed from the candidate grid map to obtain a valid drivable area of ​​the target vehicle in the current frame and output it through the input-output module.

[0144] In some implementations, the historical frame includes any first frame and second frame within the preset time window, the timestamp of the first frame is earlier than that of the second frame, and the first frame is the frame with the earliest timestamp in the point cloud accumulation of the second frame;

[0145] The processing module 402 is specifically used for:

[0146] Get the real-time speed and frame period of the target vehicle corresponding to the current frame;

[0147] Acquire the point cloud accumulation result of the second frame from the second point cloud data, and obtain the point cloud data of the first frame with the earliest timestamp in the point cloud accumulation result of the second frame;

[0148] According to the real-time vehicle speed and the frame period, motion compensation is performed on the current frame and filled into the first point cloud data of the current frame to obtain the third point cloud data.

[0149] In some implementations, the processing module 402 is specifically configured to:

[0150] Calculate the line equations of the edge point clouds corresponding to the outlines of each obstacle. Each line L between the edge point clouds iExpressed as slope k i , intercept b i , minimum vertical coordinate y imin , maximum longitudinal coordinate y imax combination of;

[0151] According to the grid granularity, traverse the longitudinal coordinate y of each grid center point n , calculate the vertical coordinate as y n When each line L i The horizontal coordinate of the intersection point is x ni , for each grid center point longitudinal coordinate y n Assign multiple intersection points lateral coordinates x nj , where j = 1, 2, ...;

[0152] According to the grid granularity, traverse the longitudinal coordinate y of each grid center point n All grids under the grid are scanned from left to right. If the center point of the leftmost grid is outside the outline area of ​​the obstacle, a value of 1 is assigned. If the center point of the leftmost grid is inside the outline area of ​​the obstacle, a value of 0 is assigned. Scan the grids to the right and repeat the assignment. During the scanning process, the horizontal coordinate x of each intersection is nj , then a negation operation is performed until all grids are assigned values ​​to obtain the drivable area and the non-drivable area.

[0153] In some implementations, the processing module 402 is specifically configured to:

[0154] Scan each row of grids G in the candidate grid map in sequence yi , find all connected first continuous grids;

[0155] Scan each column grid G ​​in the candidate grid map in turn xi , find the connected second continuous grid;

[0156] The effective drivable area is obtained according to the first continuous grid and the second continuous grid.

[0157] In some implementations, after clustering the filled third point cloud data and before extracting the contours of each obstacle, the processing module is further configured to:

[0158] From the third point cloud, point clouds with heights between the chassis of the target vehicle and the total height of the target vehicle are selected, and invalid point clouds are deleted to obtain valid point clouds constituting actual obstacles.

[0159] In some implementations, the point cloud coordinates of the third point cloud or the valid point cloud projected on the ground are set to P i (x i ,yi ); After clustering the filled third point cloud data and before extracting the contours of each obstacle, the processing module 402 is further used to:

[0160] Set the safety distance δ;

[0161] Based on the safety distance, the third point cloud or the valid point cloud is plane-expanded so that each point cloud coordinate P projected on the ground i (x i ,y i ) are replaced by a target point cloud, wherein the target point cloud includes at least one of the following items:

[0162] P i1 (x i +δ,y i +δ), P i2 (x i -δ,y i +δ), P i3 (x i -δ,y i -δ), and P i4 (x i +δ,y i -δ).

[0163] In some implementations, before acquiring the point cloud data of the target vehicle, the processing module 402 is further configured to:

[0164] Determining the detection range and grid granularity of the radar;

[0165] A grid map of the target vehicle is established according to the detection distance and the grid granularity, and the detection range of the radar covers the grid map.

[0166] The above describes the vehicle-mounted device 40 for executing the vehicle drivable area detection method in the embodiment of the present application from the perspective of modular functional entities. The following describes the vehicle-mounted device 40 for executing the vehicle drivable area detection method in the embodiment of the present application from the perspective of hardware processing. It should be noted that in the embodiment of the present application Figure 5 The entity device corresponding to the input / output module 401 in the illustrated embodiment may be an input / output unit, a transceiver, a radio frequency circuit, a communication module, an output interface, etc., and the entity device corresponding to the processing module 402 may be a processor. Figure 5 The vehicle-mounted device 40 shown may have Figure 6 The structure shown, when Figure 5 The vehicle-mounted device 40 shown has Figure 6 When the structure shown is Figure 6The processor and transceiver in the vehicle-mounted device 40 can realize the same or similar functions as the input / output module 401 and the processing module 402 provided in the above-mentioned device embodiment corresponding to the vehicle-mounted device 40. Figure 6 The memory in the storage device stores a computer program that needs to be called when the processor executes the above-mentioned vehicle drivable area detection method.

[0167] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0168] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and modules described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0169] In the several embodiments provided in the embodiments of the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or modules, which can be electrical, mechanical or other forms.

[0170] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0171] In addition, each functional module in each embodiment of the present application can be integrated into a processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The above-mentioned integrated module can be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium.

[0172] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0173] The computer program product includes one or more computer instructions. When the computer program is loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from a computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website site, a computer, a server, or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server, or data center. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a server or a data center that includes one or more available media integrations. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk Solid State Disk (SSD)), etc.

[0174] The technical solutions provided in the embodiments of the present application are introduced in detail above. The principles and implementation methods of the embodiments of the present application are explained by using specific examples in the embodiments of the present application. The description of the above embodiments is only used to help understand the methods and core ideas of the embodiments of the present application. At the same time, for those skilled in the art, according to the ideas of the embodiments of the present application, there will be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as a limitation on the embodiments of the present application.

Claims

1. A method for detecting a vehicle drivable area, characterized in that: The method comprises: Acquire target point cloud data, the target point cloud data including first point cloud data of a current frame and second point cloud data of a historical frame within a preset time window, the current frame being an image of a drivable area detected by a radar installed on the target vehicle, and the target point cloud data including a point cloud set detected by the radar; Accumulating the first point cloud data and the second point cloud data to fill the first point cloud data to obtain third point cloud data; Clustering the filled third point cloud data to obtain at least one obstacle in the current frame, each obstacle covering at least one point cloud; Extract the outline of each obstacle; Binarize the outline of each obstacle to obtain a candidate grid map, wherein the candidate grid map includes a drivable area and a non-drivable area corresponding to the target vehicle in the current frame; The non-drivable area is removed from the candidate grid map to obtain and output a valid drivable area for the target vehicle in the current frame.

2. The method according to claim 1, characterized in that: The historical frames include any first frame and second frame within the preset time window, the timestamp of the first frame is earlier than that of the second frame, and the first frame is the frame with the earliest timestamp in the point cloud accumulation of the second frame; The accumulating the first point cloud data and the second point cloud data to fill the first point cloud data to obtain third point cloud data includes: Get the real-time speed and frame period of the target vehicle corresponding to the current frame; Acquire the point cloud accumulation result of the second frame from the second point cloud data, and acquire the point cloud data of the first frame with the earliest timestamp from the point cloud accumulation result of the second frame; According to the real-time vehicle speed and the frame period, motion compensation is performed on the current frame and filled into the first point cloud data of the current frame to obtain the third point cloud data.

3. The method according to claim 2, characterized in that The step of performing raster binarization according to the outline of each obstacle to obtain a candidate raster map includes: Calculate the line equations of the edge point clouds corresponding to the outlines of each obstacle. Each line L between the edge point clouds i Expressed as slope k i , intercept b i , minimum vertical coordinate y imin , maximum longitudinal coordinate y imax combination of; According to the grid granularity, traverse the longitudinal coordinate y of each grid center point n , calculate the vertical coordinate as y n When each line L i The horizontal coordinate of the intersection point is x ni , for each grid center point longitudinal coordinate y n Assign multiple intersection points lateral coordinates x nj , where j = 1, 2, ...; According to the grid granularity, traverse the longitudinal coordinate y of each grid center point n All grids under the grid are scanned from left to right. If the center point of the leftmost grid is outside the outline area of ​​the obstacle, a value of 1 is assigned. If the center point of the leftmost grid is inside the outline area of ​​the obstacle, a value of 0 is assigned. Scan the grids to the right and repeat the assignment. During the scanning process, the horizontal coordinate x of each intersection is nj , then a negation operation is performed until all grids are assigned values ​​to obtain the drivable area and the non-drivable area.

4. The method according to any one of claims 1 to 3, characterized in that The removing the non-drivable area from the candidate grid map to obtain the valid drivable area of ​​the target vehicle in the current frame includes: Scan each row of grids G in the candidate grid map in sequence yi , find all connected first continuous grids; Scan each column grid G ​​in the candidate grid map in turn xi , find the connected second continuous grid; The effective drivable area is obtained according to the first continuous grid and the second continuous grid.

5. The method according to claim 4, characterized in that After clustering the filled third point cloud data and before extracting the contours of each obstacle, the method further includes: From the third point cloud, point clouds with heights between the chassis of the target vehicle and the total height of the target vehicle are selected, and invalid point clouds are deleted to obtain valid point clouds constituting actual obstacles.

6. The method according to claim 5, characterized in that The point cloud coordinates of the third point cloud or the valid point cloud projected on the ground are set to P i (x i ,y i ); after clustering the filled third point cloud data and before extracting the contours of each obstacle, the method further includes: Set the safety distance δ; Based on the safety distance, the third point cloud or the valid point cloud is plane-expanded so that each point cloud coordinate P projected on the ground i (x i ,y i ) are replaced by a target point cloud, wherein the target point cloud includes at least one of the following items: P i1 (x i +δ,y i +d)、P i2 (x i -d,y i +d)、P i3 (x i -d,y i -d)、and P i4 (x i +δ,y i -d).

7. The method according to claim 1, characterized in that Before acquiring the point cloud data of the target vehicle, the method further includes: Determining the detection range and grid granularity of the radar; A grid map of the target vehicle is established according to the detection distance and the grid granularity, and the detection range of the radar covers the grid map.

8. A vehicle-mounted device, characterized in that: The vehicle-mounted device comprises: An input-output module, used to obtain target point cloud data, the target point cloud data including first point cloud data of a current frame and second point cloud data of a historical frame within a preset time window, the current frame being an image of a drivable area detected by a radar installed on the target vehicle, and the target point cloud data including a point cloud set detected by the radar; A processing module, configured to accumulate the first point cloud data and the second point cloud data to fill the first point cloud data to obtain third point cloud data; Clustering the filled third point cloud data to obtain at least one obstacle in the current frame, each obstacle covering at least one point cloud; Extract the outline of each obstacle; Binarize the outline of each obstacle to obtain a candidate grid map, wherein the candidate grid map includes a drivable area and a non-drivable area corresponding to the target vehicle in the current frame; The non-drivable area is removed from the candidate grid map to obtain a valid drivable area of ​​the target vehicle in the current frame and output it through the input-output module.

9. A computer device, characterized in that: The computer device comprises: at least one processor and memory; The memory is used to store a computer program, and the processor is used to call the computer program stored in the memory to execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The method comprises instructions which, when executed on a computer, cause the computer to execute the method according to any one of claims 1 to 7.

Images