Method and device for detecting boundaries of vehicle driving area

Abstract

The present invention discloses a method and apparatus for detecting boundaries within a vehicle driving area. The method comprises: obtaining a segmentation result of a full road surface image, wherein the segmentation result of the full road surface image includes at least: a drivable area and at least one obstacle located adjacent to the drivable area; obtaining the image coordinates and categories of multiple boundary points within the drivable area based on the segmentation result of the full road surface image; and optimizing the multiple boundary points within the drivable area using the obstacle detection results to obtain an optimized result, wherein the obstacles are detected using a 2D obstacle detection model and / or a 3D obstacle detection model, and the detection result includes: position information of at least one obstacle located adjacent to the drivable area. The present invention solves the technical problem of being unable to accurately detect road surfaces within a vehicle's drivable area.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving development, and in particular to a method and device for detecting boundaries of a vehicle driving area. Background Art

[0002] Currently, the detection of autonomous driving drivable areas is mainly used to provide path planning assistance for autonomous driving. However, the existing boundary points of the drivable area have not been further optimized for the intersection points between the drivable area and obstacles, resulting in inaccurate road surface detection in the drivable area.

[0003] Currently, no effective solution has been proposed to the above-mentioned problem of being unable to accurately detect the road surface in the vehicle drivable area. Summary of the Invention

[0004] The embodiments of the present invention provide a method and apparatus for detecting a boundary of a vehicle driving area, so as to at least solve the technical problem of being unable to accurately detect a road surface in a vehicle driving area.

[0005] According to one aspect of an embodiment of the present invention, a method for detecting boundaries in a vehicle driving area is provided, comprising: obtaining a segmentation result of a full road surface map, wherein the segmentation result of the full road surface map includes at least: a drivable area and at least one obstacle located adjacent to the drivable area; obtaining image coordinates and categories of multiple boundary points within the drivable area based on the segmentation result of the full road surface map; and optimizing the multiple boundary points within the drivable area using the obstacle detection result to obtain an optimized result, wherein obstacles are detected using a 2D obstacle detection model and / or a 3D obstacle detection model, and the detection result includes: position information of at least one obstacle located adjacent to the drivable area.

[0006] Optionally, obtaining the segmentation result of the entire road surface map includes: segmenting the entire road surface map based on multiple segmentation categories, and obtaining segmentation areas formed for each segmentation category in the entire road surface map, wherein the segmentation category includes at least one of the following: road, vehicle, and pedestrian.

[0007] Optionally, based on the segmentation results of the entire road map, the image coordinates and categories of the boundary points within the drivable area are obtained, including: obtaining the area edge of each segmented area obtained by segmentation; taking the boundary points adjacent to the area edge as the boundary points of the drivable area, and obtaining the coordinates of the boundary points of the drivable area; taking the category of at least one obstacle located above the drivable area as the category of the boundary points of the drivable area.

[0008] Optionally, the obstacle detection results are used to optimize multiple boundary points within the drivable area to obtain optimization results, including: using a 2D obstacle detection model to detect at least one obstacle located at an adjacent position to the drivable area to obtain a first detection result, and using the first detection result to perform a first optimization process on multiple boundary points within the drivable area; using a 3D obstacle detection model to detect a vehicle located at an adjacent position to the drivable area to obtain a second detection result, and using the second detection result to perform a second optimization process on multiple boundary points within the drivable area.

[0009] Optionally, the first optimization process includes at least one of the following: if the 2D obstacle detection model detects that there is an overlapping part between any two obstacles, the coordinates of the non-overlapping part are used as the coordinates of the boundary point of the drivable area; if the 2D obstacle detection model detects that the distance between any obstacle and the boundary of the drivable area is less than a predetermined value, the coordinates of the point on the side of the obstacle close to the drivable area are used as the coordinates of the boundary point of the drivable area.

[0010] Optionally, the second optimization process includes at least one of the following: if the 3D obstacle detection model detects that the angular coordinates of any obstacle deviate from any direction by an angle exceeding a predetermined angle value, adjusting the coordinates of the boundary point of the drivable area according to a predetermined step size.

[0011] According to another aspect of an embodiment of the present invention, a device for detecting boundaries on a vehicle driving area is further provided, characterized in that it includes: a first acquisition module, used to obtain a segmentation result of the entire road surface map, wherein the segmentation result of the entire road surface map includes at least: a drivable area and at least one obstacle located adjacent to the drivable area; a second acquisition module, used to obtain image coordinates and categories of multiple boundary points within the drivable area based on the segmentation result of the entire road surface map; an optimization module, used to use the detection result of the obstacle to optimize the multiple boundary points within the drivable area to obtain an optimized result, wherein the obstacle is detected using a 2D obstacle detection model and / or a 3D obstacle detection model, and the detection result includes: position information of at least one obstacle located adjacent to the drivable area.

[0012] Optionally, the first acquisition module includes: a segmentation module for segmenting the entire road surface image based on multiple segmentation categories, and obtaining segmentation areas formed for each segmentation category in the entire road surface image, wherein the segmentation category includes at least one of the following: road, vehicle and pedestrian.

[0013] Optionally, the second acquisition module includes: a reading module for reading the area edge of each segmented area obtained by segmentation; a processing module for taking the boundary points adjacent to the area edge as the boundary points of the drivable area and obtaining the coordinates of the boundary points of the drivable area; and an assignment module for taking the category of at least one obstacle located above the drivable area as the category of the boundary points of the drivable area.

[0014] In an embodiment of the present invention, a segmentation result of a full road surface map is obtained, wherein the segmentation result of the full road surface map includes at least: a drivable area and at least one obstacle located adjacent to the drivable area; based on the segmentation result of the full road surface map, image coordinates and categories of multiple boundary points within the drivable area are obtained; and using the obstacle detection result, the multiple boundary points within the drivable area are optimized to obtain an optimization result, wherein obstacles are detected using a 2D obstacle detection model and / or a 3D obstacle detection model, and the detection result includes: position information of at least one obstacle located adjacent to the drivable area, thereby optimizing the multiple boundary points within the drivable area to ensure accuracy and pertinence of detection of the road surface in the vehicle drivable area, thereby achieving the technical effect of accurately detecting the road surface in the vehicle drivable area and solving the technical problem of being unable to accurately detect the road surface in the vehicle drivable area. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0016] Figure 1 is a flow chart of a method for detecting a boundary of a vehicle driving area according to an embodiment of the present invention;

[0017] Figure 2 is a schematic diagram of a device for detecting a boundary of a vehicle driving area according to an embodiment of the present invention;

[0018] Figure 3 This is a comparison diagram of the effects before and after optimization of the boundary points of the drivable area based on 2D Boundingbox processing according to an embodiment of the present invention;

[0019] Figure 4 This is a comparison diagram of the effects before and after optimization of the boundary points of the drivable area based on 3D Boundingbox processing according to an embodiment of the present invention;

[0020] Figure 5 This is a comparison diagram of the effects before and after optimization of the boundary points of the drivable area based on full-image segmentation according to an embodiment of the present invention;

[0021] Figure 6 4 is a schematic diagram of a device for detecting a boundary of a vehicle driving area according to an embodiment of the present invention. DETAILED DESCRIPTION

[0022] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0023] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0024] Example 1

[0025] According to an embodiment of the present invention, an embodiment of a method for detecting boundaries on a vehicle driving area is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0026] Figure 1 FIG. 1 is a flow chart of a method for detecting a boundary of a vehicle driving area according to an embodiment of the present invention. Figure 1 As shown, the method may include the following steps:

[0027] Step S102 : obtaining a segmentation result of the entire road surface image, wherein the segmentation result of the entire road surface image at least includes: a drivable area and at least one obstacle located adjacent to the drivable area.

[0028] In the technical solution provided in the above step S102 of the present invention, the full-image segmentation result may include: segmentation categories and results, wherein the segmentation categories include: roads, vehicles, pedestrians, cones, etc., and the rest are backgrounds, and the results include 2D pixel points of each category and the category corresponding to each pixel point.

[0029] Step S104 : Based on the segmentation result of the entire road surface image, image coordinates and categories of multiple boundary points within the drivable area are obtained.

[0030] In the technical solution provided in the above step S104 of the present invention, the image coordinates and categories of the multiple boundary points in the drivable area are the regional edges of each segmented area obtained based on the segmentation of the full road surface image.

[0031] Optionally, boundary points adjacent to the edge of the region are used as boundary points of the drivable region to obtain image coordinates of the boundary points of the drivable region.

[0032] Optionally, the category of the boundary point image of the drivable area is the category of at least one obstacle located above the drivable area.

[0033] In step S106, the obstacle detection results are used to optimize multiple boundary points within the drivable area to obtain an optimization result. The obstacles are detected using a 2D obstacle detection model and / or a 3D obstacle detection model. The detection results include: position information of at least one obstacle located at a position adjacent to the drivable area.

[0034] In the technical solution of step S106 of the present invention, the detection result may include a detection category and a detection result, wherein the detection category includes: vehicle, pedestrian, cyclist, etc., the vehicle includes the full vehicle frame and the rear frame, and the detection result includes: the 2D Boundingbox coordinates and 3D Boundingbox coordinates of the obstacle of each category.

[0035] In the above steps S102 to S106 of the present application, a segmentation result of the entire road surface map is obtained, wherein the segmentation result of the entire road surface map includes at least: a drivable area and at least one obstacle located adjacent to the drivable area; based on the segmentation result of the entire road surface map, the image coordinates and categories of multiple boundary points in the drivable area are obtained; and using the obstacle detection results, the multiple boundary points in the drivable area are optimized to obtain an optimization result, wherein the obstacles are detected using a 2D obstacle detection model and / or a 3D obstacle detection model, and the detection results include: position information of at least one obstacle located adjacent to the drivable area, thereby optimizing the multiple boundary points in the drivable area to ensure the accuracy and pertinence of the detection of the road surface in the vehicle drivable area, thereby achieving the technical effect of accurately detecting the road surface in the vehicle drivable area and solving the technical problem of being unable to accurately detect the road surface in the vehicle drivable area.

[0036] The above method of this embodiment is further introduced below.

[0037] As an optional embodiment, step S102, obtaining the segmentation results of the entire road surface map, also includes: segmenting the entire road surface map based on multiple segmentation categories, and obtaining segmentation areas formed for each segmentation category in the entire road surface map, wherein the segmentation category includes at least one of the following: road, vehicle, and pedestrian.

[0038] Optionally, the segmentation result includes 2D pixel points of each category and the category corresponding to each pixel point.

[0039] As an optional implementation method, step S104 obtains the image coordinates and categories of the boundary points within the drivable area based on the segmentation results of the entire road map, including: obtaining the area edge of each segmented area obtained by segmentation; using the boundary points adjacent to the area edge as the boundary points of the drivable area, and obtaining the coordinates of the boundary points of the drivable area; and using the category of at least one obstacle located above the drivable area as the category of the boundary points of the drivable area.

[0040] In this embodiment, based on the segmentation results based on the full road map, the area frame formed by each segmentation category is obtained, and then the coordinates of the pixel points of each category are segmented to obtain the area frame coordinates of the segmented area of ​​that category. The area frames are sorted from small to large, and finally the points in each area frame are scanned row by row from bottom to top according to the image height. The points in the row that belong to the area frame and are determined as the boundary points of the drivable area in the previous row are used as the boundary points of the drivable area, and the obstacle category above the drivable area is used as the category of the updated drivable area boundary point.

[0041] As an optional implementation method, step S106 uses the detection results of the obstacle to optimize the multiple boundary points within the drivable area to obtain an optimization result, including: using a 2D obstacle detection model to detect at least one obstacle located at an adjacent position to the drivable area to obtain a first detection result, and using the first detection result to perform a first optimization process on the multiple boundary points within the drivable area; using a 3D obstacle detection model to detect a vehicle located at an adjacent position to the drivable area to obtain a second detection result, and using the second detection result to perform a second optimization process on the multiple boundary points within the drivable area.

[0042] In this embodiment, 2D Boundingbox is used to detect obstacles in the drivable area, and 3D Boundingbox is used to detect other contents in the drivable area. The 2D Boundingbox detection result is used to optimize the pixel-level segmentation result of the drivable area for the rear frame, and the 3D Boundingbox detection result is used to optimize the pixel-level segmentation result of the drivable area for the vehicle.

[0043] As an optional embodiment, in step S106, the first optimization process includes at least one of the following: if the 2D obstacle detection model detects that any two obstacles have overlapping portions, the coordinates of the non-overlapping portions are used as the coordinates of the boundary point of the drivable area; if the 2D obstacle detection model detects that the distance of any obstacle from the boundary of the drivable area is less than a predetermined value, the coordinates of the point on the side of the obstacle closest to the drivable area are used as the coordinates of the boundary point of the drivable area.

[0044] In this embodiment, the first optimization process detects obstacles in the drivable area through a 2D obstacle detection model to determine the coordinates of boundary points of the drivable area.

[0045] For example, if multiple obstacles are detected to be partially overlapping, the coordinates of the boundary point of the drivable area are the coordinates of the non-overlapping part; if an obstacle is detected to be close to the boundary of the drivable area at a distance less than a predetermined value, the coordinates of the boundary point of the drivable area are the coordinates of the point on the side of the obstacle close to the drivable area.

[0046] As an optional embodiment, in step S106, the second optimization process includes at least one of the following: if the 3D obstacle detection model detects that the angular coordinates of any obstacle deviate from any direction by an angle exceeding a predetermined angle value, the coordinates of the boundary point of the drivable area are adjusted according to a predetermined step size.

[0047] In this embodiment, the second optimization process detects obstacles in the drivable area through a 3D obstacle detection model, and updates and adjusts the coordinates of the boundary points of the drivable area according to a predetermined step size.

[0048] This embodiment obtains a segmentation result of a full road surface map, wherein the segmentation result of the full road surface map includes at least: a drivable area and at least one obstacle located adjacent to the drivable area; based on the segmentation result of the full road surface map, image coordinates and categories of multiple boundary points in the drivable area are obtained; and using the obstacle detection result, multiple boundary points in the drivable area are optimized to obtain an optimization result, wherein obstacles are detected using a 2D obstacle detection model and / or a 3D obstacle detection model, and the detection result includes: position information of at least one obstacle located adjacent to the drivable area, thereby optimizing multiple boundary points in the drivable area to ensure accuracy and pertinence of detection of the road surface in the vehicle drivable area, thereby achieving the technical effect of accurately detecting the road surface in the vehicle drivable area and solving the technical problem of being unable to accurately detect the road surface in the vehicle drivable area.

[0049] Example 2

[0050] The technical solutions of the embodiments of the present invention are described below with reference to preferred implementation methods.

[0051] Currently, the detection of the drivable area for autonomous driving is mainly used to provide path planning assistance for autonomous driving. It can detect the entire road surface or only extract partial road information. However, if there are obstacles above the drivable area, such as vehicles or pedestrians, the segmentation result of the drivable area category based solely on full-image segmentation is far from enough. If the boundary point segmentation between the obstacle and the drivable area is relatively rough, this will affect the road surface detection and path planning of autonomous driving based on the drivable area. Therefore, in order to obtain more realistic drivable area boundary point information, it is very important to further optimize the drivable area boundary points.

[0052] Existing methods for obtaining drivable area boundary points primarily rely on full-image semantic segmentation results. These methods include traditional machine learning algorithms, such as texture extraction, edge extraction, vanishing point extraction, and support vector machine detection classifiers. There are also deep learning-based drivable area detection algorithms, such as DeepLabV3+. However, the segmentation points where the drivable area intersects with obstacles have not been further optimized, resulting in inaccurate drivable area road surface detection.

[0053] Therefore, in order to overcome the above problems, in a related technology, a semantic segmentation optimization method for autonomous driving images is proposed, which is characterized by being divided into the following steps: 1) This method designs an AAM module that uses labels to assist activation, and corrects the features extracted by the network through segmentation labels, so that the features extracted from similar objects have approximately the same values; 2) The AAM module is integrated between the encoder and decoder of the segmentation model, and a model with better performance than the baseline model is obtained through training, which is called the teacher network; 3) Through knowledge transfer, the knowledge learned by the teacher network based on the AAM module is transferred to the segmentation model to improve its segmentation performance, thereby solving the technical problem of being unable to effectively mine the information of the segmentation label, improving the performance of the segmentation model, and without modifying the network structure, it has strong application value.

[0054] However, this application proposes a method for optimizing the boundary points of the drivable area based on full-graph segmentation. This method can make the boundary points of the drivable area further fit the obstacle targets, so as to accurately perform road surface detection and path planning based on the drivable area. Moreover, the optimization algorithm has strong adaptability, low algorithm complexity, and is easy to implement.

[0055] In this embodiment, a flow chart of a method for optimizing the boundary points of a drivable area is proposed, such as Figure 2 As shown, Figure 2 This is a flow chart of a method for optimizing boundary points of a drivable area according to an embodiment of the present invention.

[0056] S202: Obtain the 2D image coordinates and categories of the boundary points of the drivable area based on the full image segmentation result.

[0057] According to the semantic segmentation results of the whole image of each category, obtain the region frame SEG_BBox formed by each segmentation category, and set the coordinates of the segmentation pixel points of this category to (x i ,y i ), then the SEG_BBox coordinates of the segmented area of ​​this category are:

[0058] SEG_BBox xmin =min(x i )SEG_BBox xmax =max(x i )

[0059] SEG_BBox ymin =min(y i )SEG_BBox ymax =max(y i )

[0060] Set SEG_BBox to SEG_BBox yminSort from small to large, scan the points in each SEG_BBox row by row from bottom to top according to the image height, and use the points in the row that belong to the SEG_BBox and are determined as the boundary points of the drivable area in the previous row as the boundary points of the drivable area.

[0061] The obstacle category above the drivable area is used as the category of the updated drivable area boundary point.

[0062] S204, based on 2D obstacle target detection, the 2D Boundingbox detection results are used to optimize the pixel-level segmentation results of the drivable area. Here, the vehicle frame is the rear frame. The boundary points before and after optimization are as follows: Figure 3 shown.

[0063] Optionally, if there is overlap between two obstacles, set 2D_BBox ymin The coordinates of the smaller non-overlapping part are assigned to the corresponding boundary point of the drivable area. The non-overlapping of two obstacles is determined as follows:

[0064] box1.1>box2.r||box1.r<box2.1||box1.t>box2.b

[0065] Among them, box1 and box2 are the 2D Boundingbox of two obstacles, and l is 2D_BBox xmin , r is 2D_BBox xmax , t is 2D_BBox ymin , b is 2D_BBox ymax .

[0066] Optionally, when the obstacle2D_BBox ymax If the bottom edge point is less than 10 pixels away from the boundary point of the drivable area, use obstacle 2D_BBox ymax The bottom edge point is assigned to the corresponding boundary point of the drivable area.

[0067] S206: For the boundary points of the drivable area that have not been optimized, based on 3D obstacle target detection, the pixel-level segmentation results of the drivable area (for vehicles) are optimized using the 3D sounding box detection results, such as Figure 4 shown.

[0068] Optionally, if the yaw angle of the 3D BoundingBox obstacle is to the left, that is, if the following conditions are met:

[0069] box3d.lower.1t.x<box3d.lower.1b.x&&box3d.lower.1t.x<left

[0070] Among them, lower is the bottom surface 2D_BBox of 3D BoundingBox, lt is the upper left point, and lb is the lower left point. Set the tail box 2D_BBox xmin The coordinate of left is left, and the boundary point of the drivable area corresponding to left is DRI[left]. The boundary points of the drivable area within the range of (box3d.lower.lt.x, left) are optimized using the corresponding DRI[box3d.lower.lt.x] point with a certain step size. The step size formula is as follows:

[0071] step=(DRI[left]-box3d.lower.1t.y) / (left-box3d.lower.1t.x)

[0072] Then the optimization formula for the boundary points of the drivable area within the range of (box3d.lower.lt.x, left) is:

[0073] DRI[t+box3d.lower.1r.x]=DRI[box3d.lower.1r.x]+t*step,

[0074] t∈[box3d.lower.1r.x,left]

[0075] Optionally, if the yaw angle of the 3D BoundingBox obstacle is to the right, that is, if the following conditions are met:

[0076] box3d.lower.rt.x>box3d.lower.rb.x&&box3d.lower.rt.x>right

[0077] Where lower is the bottom 2D_BBox of the 3D BoundingBox, rt is the upper right point, and rb is the lower right point. xmax The coordinate of right is right, and the boundary point of the drivable area corresponding to right is DRI[right]. The boundary points of the drivable area within the range of (right, box3d.lower, rt.x) are optimized with a certain step size using the DRI[right] point optimized for the corresponding tail box. The step size formula is as follows:

[0078] step=(box3d.lower.rt.y-DRI[right]) / (box3d.lower.rt.x-right)

[0079] Then the optimization formula for the boundary points of the drivable area within the range of (right, box3d.lower.rt.x) is:

[0080] DRI[t+right]=DRI[right]+t*step,t∈[right,box3d.lower.rt.x]

[0081] Based on the full image segmentation results, the 2D image coordinates and categories of the boundary points of the drivable area are obtained. Then, based on the 2D obstacle target detection, the 2D Boundingbox detection results are used to optimize the pixel-level segmentation results of the drivable area. Finally, based on the 3D obstacle target detection, the 3D Boundingbox detection results are used to optimize the pixel-level segmentation results of the drivable area. Figure 5 As shown, Figure 5 This is a comparison diagram of the effects before and after optimization of the boundary points of the drivable area based on full-image segmentation according to an embodiment of the present invention, where the black line represents before optimization and the red line represents after optimization.

[0082] In this embodiment, by obtaining the segmentation results of the entire road surface map, and then based on the segmentation results of the entire road surface map, obtaining the image coordinates and categories of multiple boundary points in the drivable area, and finally using the obstacle detection results, optimizing the multiple boundary points in the drivable area to obtain the optimization results, thereby achieving the technical effect of accurately detecting the road surface in the vehicle drivable area, and solving the technical problem of being unable to accurately detect the road surface in the vehicle drivable area.

[0083] Example 3

[0084] According to an embodiment of the present invention, a device for detecting a boundary of a vehicle driving area is also provided. It should be noted that the device for detecting a boundary of a vehicle driving area can be used to execute the method for detecting a boundary of a vehicle driving area in embodiment 1.

[0085] Figure 6 FIG. 1 is a schematic diagram of a device for detecting a boundary of a vehicle driving area according to an embodiment of the present invention. Figure 6 As shown, the device 600 for detecting the boundary of the vehicle driving area may include: a first acquisition module 601 , a second acquisition module 602 , and an optimization module 603 .

[0086] The first acquisition module 601 is configured to acquire a segmentation result of the entire road surface image, wherein the segmentation result of the entire road surface image at least includes: a drivable area and at least one obstacle located adjacent to the drivable area.

[0087] The second acquisition module 602 is used to acquire the image coordinates and categories of multiple boundary points within the drivable area based on the segmentation result of the entire road map.

[0088] Optimization module 603 is configured to utilize the obstacle detection results to optimize multiple boundary points within the drivable area to obtain an optimization result. Obstacle detection is performed using a 2D obstacle detection model and / or a 3D obstacle detection model. The detection result includes location information of at least one obstacle located adjacent to the drivable area.

[0089] Optionally, the first acquisition module includes: a sub-segmentation module, which is used to segment the entire road surface image based on multiple segmentation categories, and obtain segmentation areas formed for each segmentation category in the entire road surface image, wherein the segmentation category includes at least one of the following: road, vehicle and pedestrian.

[0090] Optionally, the second acquisition module includes: a sub-reading module for reading the area edge of each segmented area obtained by segmentation; a sub-processing module for taking the boundary points adjacent to the area edge as the boundary points of the drivable area and obtaining the coordinates of the boundary points of the drivable area; a sub-assignment module for taking the category of at least one obstacle located above the drivable area as the category of the boundary points of the drivable area.

[0091] In this embodiment, the segmentation results of the entire road surface map are obtained by the first acquisition module. Then, based on the segmentation results of the entire road surface map, the second acquisition module obtains the image coordinates and categories of multiple boundary points within the drivable area. Finally, the optimization module uses the obstacle detection results to optimize the multiple boundary points within the drivable area to obtain the optimization results, thereby achieving the technical effect of accurately detecting the road surface in the vehicle drivable area and solving the technical problem of being unable to accurately detect the road surface in the vehicle drivable area.

[0092] The serial numbers of the above embodiments of the present invention are for description only and do not represent the advantages or disadvantages of the embodiments.

[0093] In the above embodiments of the present invention, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0094] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only exemplary. For example, the division of units can be a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0095] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple units. Some or all of the units may be selected to achieve the purpose of the present embodiment according to actual needs.

[0096] In addition, the functional units in the various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0097] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server or network device, etc.) to execute all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk, etc. Various media that can store program codes.

[0098] The above are only preferred embodiments of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for detecting a boundary of a vehicle driving area, characterized in that: include: Obtaining a segmentation result of the entire road map, wherein the segmentation result of the entire road map at least includes: a drivable area and at least one obstacle located adjacent to the drivable area; Based on the segmentation result of the full road map, obtaining image coordinates and categories of multiple boundary points within the drivable area; Optimizing a plurality of boundary points within the drivable area using the obstacle detection results to obtain an optimization result, wherein the obstacles are detected using a 2D obstacle detection model and / or a 3D obstacle detection model, and the detection result includes: position information of at least one obstacle located adjacent to the drivable area; wherein, optimizing a plurality of boundary points within the drivable area using the obstacle detection result to obtain an optimization result includes: detecting obstacles in the drivable area using the 2D obstacle detection model to obtain a first detection result; determining coordinates of boundary points of the drivable area based on the first detection result; and adjusting the coordinates of the boundary points of the drivable area based on a predetermined step size to obtain the optimization result; Determining the coordinates of the boundary point of the drivable area based on the first detection result includes: in response to the first detection result indicating that any two obstacles have overlapping portions, determining the coordinates of the non-overlapping portions of the any two obstacles as the coordinates of the boundary point of the drivable area; or, in response to the first detection result indicating that the distance between any obstacle and the boundary of the drivable area is less than a predetermined value, determining the coordinates of a point on the obstacle close to one side of the drivable area as the coordinates of the boundary point of the drivable area; The optimization result is obtained by adjusting the coordinates of the boundary points of the drivable area based on a predetermined step size, including: detecting the obstacle through the 3D obstacle detection model to obtain a second detection result of the detection result; in response to the second detection result that the angular coordinates of any obstacle deviate from any direction by an angle exceeding a predetermined angle value, adjusting the coordinates of the boundary points of the drivable area according to the predetermined step size to obtain the optimization result.

2. The method according to claim 1, characterized in that Obtaining a segmentation result of a full road surface image includes: segmenting the full road surface image based on multiple segmentation categories, and obtaining segmented areas formed for each segmentation category in the full road surface image, wherein the segmentation category includes at least one of the following: road, vehicle, and pedestrian.

3. The method according to claim 2, characterized in that Based on the segmentation result of the full road map, the image coordinates and categories of the boundary points within the drivable area are obtained, including: Get the region edge of each segmented region obtained by segmentation; Taking a boundary point adjacent to the edge of the area as a boundary point of the drivable area, and obtaining the coordinates of the boundary point of the drivable area; The category of the at least one obstacle located above the drivable area is used as the category of the boundary point of the drivable area.

4. The method according to any one of claims 1 to 3, characterized in that Utilizing the obstacle detection results, multiple boundary points within the drivable area are optimized to obtain optimization results, including: Using the 2D obstacle detection model to detect at least one obstacle located adjacent to the drivable area to obtain a first detection result, and using the first detection result to perform a first optimization process on a plurality of boundary points within the drivable area; The 3D obstacle detection model is used to detect the vehicle located at a position adjacent to the drivable area to obtain a second detection result, and the second detection result is used to perform a second optimization process on multiple boundary points within the drivable area.

5. The method according to claim 4, characterized in that The first optimization process includes at least one of the following: If the 2D obstacle detection model detects that any two obstacles have overlapping parts, the coordinates of the non-overlapping parts are used as the coordinates of the boundary points of the drivable area; If the 2D obstacle detection model detects that the distance of any obstacle close to the boundary of the drivable area is less than a predetermined value, the coordinates of the point of the obstacle close to one side of the drivable area are used as the coordinates of the boundary point of the drivable area.

6. The method according to claim 4, characterized in that The second optimization process includes at least one of the following: If the 3D obstacle detection model detects that the angular coordinates of any obstacle deviate from any direction by an angle exceeding the predetermined angle value, the coordinates of the boundary point of the drivable area are adjusted according to the predetermined step size.

7. A device for detecting the boundary of a vehicle driving area, characterized in that: include: A first acquisition module is configured to acquire a segmentation result of a full road map, wherein the segmentation result of the full road map includes at least: a drivable area and at least one obstacle located adjacent to the drivable area; A second acquisition module is configured to acquire image coordinates and categories of a plurality of boundary points within the drivable area based on the segmentation result of the full road map; an optimization module, configured to optimize a plurality of boundary points within the drivable area using the obstacle detection results to obtain an optimization result, wherein the obstacles are detected using a 2D obstacle detection model and / or a 3D obstacle detection model, and the detection result includes: position information of at least one obstacle located adjacent to the drivable area; The optimization module is further configured to detect obstacles in the drivable area using the 2D obstacle detection model to obtain a first detection result; determine the coordinates of the boundary points of the drivable area based on the first detection result; and adjust the coordinates of the boundary points of the drivable area based on a predetermined step size to obtain the optimization result. The device is further configured to determine the coordinates of a boundary point of the drivable area based on the first detection result by: in response to the first detection result indicating that any two obstacles have an overlapping portion, determining the coordinates of the non-overlapping portion of the any two obstacles as the coordinates of the boundary point of the drivable area; or, in response to the first detection result indicating that the distance between any obstacle and the boundary of the drivable area is less than a predetermined value, determining the coordinates of a point on the obstacle close to one side of the drivable area as the coordinates of the boundary point of the drivable area; The device is also used to adjust the coordinates of the boundary points of the drivable area based on a predetermined step size through the following steps to obtain the optimization result: detecting the obstacle through the 3D obstacle detection model to obtain a second detection result of the detection result; in response to the second detection result that the angular coordinates of any obstacle deviate from any direction by an angle exceeding a predetermined angle value, adjusting the coordinates of the boundary points of the drivable area according to the predetermined step size to obtain the optimization result.

8. The device according to claim 7, characterized in that The first acquisition module includes: a sub-segmentation module, which is used to segment the full road map based on multiple segmentation categories and obtain segmentation areas formed for each segmentation category in the full road map, wherein the segmentation category includes at least one of the following: road, vehicle and pedestrian.

9. The device according to claim 8, characterized in that The second acquisition module includes: A sub-reading module is used to read the region edge of each segmented region obtained by segmentation; a sub-processing module, configured to use boundary points adjacent to the edge of the area as boundary points of the drivable area, and obtain coordinates of the boundary points of the drivable area; The sub-assignment module is used to use the category of the at least one obstacle located above the drivable area as the category of the boundary point of the drivable area.

10. A vehicle, characterized in that: The method for detecting the boundary of a vehicle driving area includes the method described in any one of claims 1 to 6.

Images