Travelable area marking method and device and computer storage medium

Abstract

The present application relates to the field of automatic driving, and more particularly, to a drivable area labeling method, a drivable area labeling device implementing the method, and a computer storage medium implementing the method. The method comprises the following steps: generating a road edge marking line and a road internal forbidden area marking line on a BEV image based on manual marking information; performing triangulation processing on the point set forming the road edge marking line by using a triangulation algorithm to obtain a triangular mesh; screening the edges forming the triangular mesh to obtain the outer contour of the drivable area conforming to the road structure rule; dividing a drivable area and a non-drivable area in the outer contour of the drivable area by using an image segmentation method based on the marking line of the forbidden area in the road; and generating travelable area marking information based on the division result.

Description

Technical Field

[0001] The present application relates to the field of autonomous driving, and more specifically to a drivable area labeling method, a drivable area labeling device for implementing the method, and a computer storage medium for implementing the method. Background Art

[0002] In the field of autonomous driving technology, accurate identification of drivable areas is one of the core functions of the perception system. Building an efficient drivable area segmentation model relies on rich, high-quality drivable area annotation data. The accuracy and completeness of the annotation data have a decisive impact on the model performance.

[0003] At present, the labeling of drivable areas mainly relies on manual operations, which not only involves high time and labor costs, but also has a high risk of errors due to the complexity of the labeling process, including wrong labeling and omissions. Automated labeling methods, such as the traditional concave hull algorithm, can provide assistance in some cases, but the labeling results they generate often deviate from the actual road structure and are too dependent on curb information, resulting in large errors and reducing the reliability and practicality of the generated data.

[0004] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present application, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0005] In order to solve or at least alleviate one or more of the above problems, the following technical solutions are provided. The embodiments of the present application provide a drivable area labeling method, a drivable area labeling device implementing the method, and a computer storage medium implementing the method, which can realize automatic labeling of drivable areas based on incomplete manual labeling information, while improving the reliability of the labeling data.

[0006] According to a first aspect of the present application, a method for marking a drivable area is provided, the method comprising the following steps: based on manual marking information, generating curb marking lines and road internal restricted zone marking lines on a BEV image; using a triangulation algorithm to triangulate the point set constituting the curb marking lines to obtain a triangular mesh; screening the edges constituting the triangular mesh to obtain an outer contour of the drivable area that conforms to road structure rules; based on the road internal restricted zone marking lines, using an image segmentation method to divide the outer contour of the drivable area into a drivable area and a non-drivable area; and generating drivable area marking information based on the division result.

[0007] As an alternative or supplement to the above scheme, in a drivable area marking method according to an embodiment of the present application, the road internal restricted area marking lines include one or more of flower bed marking lines, diversion area marking lines, no-parking area marking lines, and red warning area marking lines.

[0008] As an alternative or supplement to the above solution, in a drivable area marking method according to an embodiment of the present application, the triangulation algorithm is a Delauny triangulation algorithm, and the circumscribed circle of any triangle in the triangular mesh does not contain vertices of other triangles.

[0009] As an alternative or supplement to the above scheme, in a drivable area marking method according to an embodiment of the present application, screening the edges constituting the triangular mesh includes: dividing the edges constituting the triangular mesh into the following three categories based on the number of adjacent triangles: internal edges with a number of adjacent triangles greater than 1, boundary edges with a number of adjacent triangles of 1, and degenerate edges with a number of adjacent triangles of 0; and eliminating boundary edges with a length greater than a road width threshold.

[0010] As an alternative or supplement to the above scheme, in a drivable area labeling method according to an embodiment of the present application, screening the edges constituting the triangular mesh also includes: adding boundary edges whose lengths are greater than the road width threshold to a screening queue; selecting a boundary edge from the screening queue, and deleting the selected boundary edge from the screening queue; obtaining the other two edges in the adjacent triangles of the selected boundary edge, reducing the number of adjacent triangles of the other two edges by one, and reclassifying the other two edges based on the updated number of adjacent triangles; and updating the screening queue based on the reclassification result, and repeating the above operations for the updated screening queue until the screening queue is empty.

[0011] As an alternative or supplement to the above scheme, in a drivable area marking method according to an embodiment of the present application, screening the edges constituting the triangular mesh based on road structure rules also includes: if the first endpoint of the first boundary edge belongs to the first curb marking line, the second endpoint belongs to the second curb marking line, and the first endpoint is not the line endpoint of the first curb marking line, then determining the line endpoint closest to the second endpoint on the first curb marking line; and moving the first endpoint along the first curb marking line toward the determined line endpoint until the first endpoint moves to the determined line endpoint, or the first boundary edge intersects the restricted area marking line inside the road, and stops, and updating the moved first boundary edge to a new boundary edge.

[0012] As an alternative or supplement to the above scheme, in a drivable area marking method according to an embodiment of the present application, screening the edges constituting the triangular mesh also includes: if the second boundary edge is not included in any curb marking line, and the other two edges in the adjacent triangle of the second boundary edge are included in the third curb marking line, then the second boundary edge is recorded as a degenerate edge, and the third curb marking line is recorded as a boundary edge.

[0013] As an alternative or supplement to the above scheme, in a drivable area labeling method according to an embodiment of the present application, the edges constituting the triangular mesh are screened to obtain the outer contour of the drivable area that conforms to the road structure rules, including: recording the boundary edges retained after screening as valid boundary edges; and generating the outer contour of the drivable area based on the valid boundary edges.

[0014] As an alternative or supplement to the above solution, in a method for marking a drivable area according to an embodiment of the present application, the method further includes: closing the marking line of the prohibited area inside the road.

[0015] As an alternative or supplement to the above scheme, in a drivable area marking method according to an embodiment of the present application, closing the restricted area marking line inside the road includes: if the distance between the two endpoints of the restricted area marking line inside the road is less than a distance threshold, connecting the two endpoints with a straight line; and / or if the restricted area marking line inside the road is a U-shaped line, connecting the two endpoints of the U-shaped line with a straight line.

[0016] As an alternative or supplement to the above scheme, in a method for marking a drivable area according to an embodiment of the present application, the image segmentation method is a region growing algorithm, and based on the marking lines of the restricted area inside the road, the image segmentation method is used to divide the drivable area and the non-drivable area within the outer contour of the drivable area, including: gridding the BEV image; using the outer contour of the drivable area and the marking lines of the restricted area inside the closed road as growth stop conditions, and starting from the grid where the vehicle is located to grow towards the adjacent grids.

[0017] As an alternative or supplement to the above scheme, in a drivable area labeling method according to an embodiment of the present application, generating drivable area labeling information based on the division results includes: marking the drivable area divided in the BEV image as a drivable area, and marking other areas as non-drivable areas.

[0018] According to a second aspect of the present application, a drivable area marking device is provided, comprising: a memory; a processor; and a computer program stored in the memory and executable on the processor, wherein the execution of the computer program enables any one of the drivable area marking methods described in the first aspect of the present application to be executed.

[0019] According to a third aspect of the present application, a computer storage medium is provided, wherein the computer storage medium includes instructions, and the instructions, when run, execute any one of the drivable area marking methods described in the first aspect of the present application.

[0020] The drivable area labeling scheme according to one or more embodiments of the present application realizes the automatic labeling of the drivable area suitable for the field of autonomous driving. It can realize the construction of a concave hull that complies with the road structure rules (that is, the outer contour of the drivable area that complies with the road structure rules), and supports the automatic completion of missing road structures. At the same time, it has low requirements on the accuracy of manual labeling, and reliable drivable area labeling data can still be generated even when the quality of manual labeling data is poor. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The above and / or other aspects and advantages of the present application will become clearer and easier to understand through the following description of various aspects in conjunction with the accompanying drawings, in which the same or similar units are represented by the same reference numerals. In the accompanying drawings:

[0022] Figure 1 is a schematic flow chart of a drivable area marking method 10 according to one or more embodiments of the present application;

[0023] Figure 2 An exemplary schematic diagram of a BEV image with annotation information added according to one or more embodiments of the present application;

[0024] Figure 3 is an exemplary schematic diagram of a triangulation result according to one or more embodiments of the present application;

[0025] Figure 4 is an exemplary schematic diagram of updating a boundary edge according to one or more embodiments of the present application;

[0026] Figure 5 is an exemplary schematic diagram of updating a boundary edge according to one or more embodiments of the present application;

[0027] Figure 6 is an exemplary schematic diagram of the outer contour of a drivable area according to one or more embodiments of the present application;

[0028] Figure 7 An exemplary schematic diagram of marking information of a drivable area according to one or more embodiments of the present application; and

[0029] Figure 8 is a schematic block diagram of a drivable area marking device 80 according to one or more embodiments of the present application. DETAILED DESCRIPTION

[0030] The description of the following specific embodiments is merely exemplary in nature and is not intended to limit the disclosed technology or the application and use of the disclosed technology. In addition, it is not intended to be bound by any express or implied theory presented in the aforementioned technical field, background technology or the following specific embodiments.

[0031] In the following detailed description of the embodiments, many specific details are set forth in order to provide a more thorough understanding of the disclosed technology. However, it is apparent to one of ordinary skill in the art that the disclosed technology can be practiced without these specific details. In other instances, well-known features are not described in detail to avoid unnecessarily complicating the description.

[0032] Terms such as "comprising" and "including" indicate that in addition to the units and steps directly and clearly stated in the specification, the technical solution of the present application does not exclude the situation of having other units and steps that are not directly or clearly stated. Terms such as "first" and "second" do not indicate the order of units in terms of time, space, size, etc., but are only used to distinguish each unit. The technology of the present application is generally used in electric vehicles, which include but are not limited to pure electric vehicles (BEV), hybrid electric vehicles (HEV), fuel cell vehicles (FCEV), etc.

[0033] An automated driving system (ADS) refers to a system that will continuously perform all dynamic driving tasks (DDT) within its own operational domain design (ODD). That is, the machine system is allowed to fully assume the task of automatic vehicle control only under the specified appropriate driving scenario conditions, that is, the system is activated when the vehicle meets the ODD conditions, and the system replaces the human driver as the driving subject of the vehicle. Among them, the dynamic driving task DDT refers to the continuous lateral (left and right steering) and longitudinal motion control (acceleration, deceleration, constant speed) of the vehicle, as well as the detection and response to targets and events in the vehicle's driving environment. The design operating range ODD refers to the conditions under which the automated driving system can operate safely, and the conditions set may include geographic location, road type, speed range, weather, time, national and local traffic laws and regulations, etc.

[0034] The lane recognition model of the autonomous driving system needs to be trained using the manually labeled information of the roadside and no-go areas (for example, flower beds, diversion areas, no-parking areas, red alert areas, etc.). Since the accuracy of the above-mentioned manually labeled information required by the lane recognition model is not high and the area is not closed, it is not enough to support the training of the drivable area segmentation model, and additional labeling steps are still required. In this regard, some technical solutions rely on additional manual labeling to generate drivable area labeling information. However, such manual labeling methods often have the following pain points: poor effect accuracy, long labeling cycle, and high time consumption; they are strongly related to the labeling rules. When the labeling rules need to be changed or new requirements are added, manual labeling needs to be repeated, which consumes a lot of manpower and material resources. Some other technical solutions are based on traditional concave hull algorithms to achieve automatic labeling. Since such solutions do not take into account the road structure requirements, the labeling failure rate is high and the reliability is poor. In this regard, the drivable area labeling scheme according to the present application aims to construct a concave hull based on the existing labeling data (i.e., the manual labeling information of the roadside and the prohibited area) through a triangulation algorithm combined with the road structure characteristics to realize the automatic labeling of the drivable area, thereby saving the labeling cost and improving the reliability of the labeling data. In the following, various exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings.

[0035] Referring to the accompanying drawings, Figure 1 1 is a schematic flow chart of a drivable area marking method 10 according to one or more embodiments of the present application.

[0036] like Figure 1 As shown, in step S110, based on the manual annotation information, the roadside annotation lines and the road internal restricted zone annotation lines are generated on the BEV image.

[0037] During the training data collection process, the LiDAR (Light Detection and Ranging) system on the test vehicle is used to scan the surrounding environment to obtain point cloud data. The original point cloud data is a collection of a large number of discrete points in three-dimensional space. After pre-processing operations such as denoising, filtering, and coordinate transformation, the ground point cloud can be separated from the non-ground point cloud through a classification algorithm, and the non-ground point cloud data can be projected from the three-dimensional space to a two-dimensional plane so as to be converted to a bird's eye view (BEV) perspective, thereby generating a two-dimensional BEV image. Compared with the original point cloud data, the two-dimensional BEV image provides two-dimensional plane information observed from a bird's eye view, and pays more attention to the presentation of the image and the visualization of information, so it is more suitable for manual annotation.

[0038] In one possible embodiment, when performing manual annotation, an image annotation tool can be used to directly draw the roadside annotation line and the road internal restricted zone annotation line on the BEV image. Exemplarily, the road internal restricted zone annotation line includes one or more of the flower bed annotation line, the diversion zone annotation line, the no-parking zone annotation line, and the red warning zone annotation line. In another possible embodiment, text annotations can also be added to the BEV image (for example, the roadside point is marked as "1" and the road internal restricted zone is marked as "2"), and the roadside annotation line and the road internal restricted zone annotation line are generated based on the text annotation results. Usually, the roadside annotation line is embodied in the form of a continuous line along the edge of the road, while the road internal restricted zone annotation line is embodied in the form of a frame along the boundary line of the restricted zone or in the form of a line inside the restricted zone. It should be noted that, limited by the precision and accuracy of manual annotation, the roadside annotation line and the road internal restricted zone annotation line inevitably have omissions and mislabeling situations, for example, the flower bed annotation line is in an unclosed form, and there is a gap in the roadside annotation line. In this regard, steps S120 - S130 described below can realize automatic completion of the road structure to facilitate the division of the drivable area.

[0039] In step S120, a triangulation algorithm is used to triangulate the point set constituting the roadside marking line to obtain a triangular mesh.

[0040] The triangulation algorithm is a process of generating a set of triangles (i.e., a triangular mesh) based on a given set of points. It is mainly used in computer graphics, geographic information systems, and other fields. In two-dimensional space, the triangulation algorithm usually requires that the endpoints of all triangles in the triangular mesh just form a given set of points, the sides of any two triangles do not intersect, and the set of all triangles forms the convex hull of the point set. Therefore, the outer contour of the triangular mesh obtained by the triangulation algorithm can form the convex hull of the point set.

[0041] Delaunay triangulation is a special triangulation method that requires that the circumscribed circle of any triangle in the triangular mesh does not contain other vertices in the point set (i.e., it satisfies the empty circle property). The triangular mesh obtained by the Delaunay triangulation algorithm has the following characteristics: uniqueness, that is, no matter where the construction starts in the region, the result is consistent; maximizing the minimum angle, that is, among all possible triangulations, the minimum angle of the triangle formed by Delaunay triangulation is the largest, which makes the triangular mesh closer to regularization and is conducive to subsequent analysis and processing; regionality, that is, when adding, deleting or moving a vertex, it will only affect the adjacent triangles, which makes Delaunay triangulation more efficient when processing dynamic data; convex hull, that is, the outermost boundary of the triangular mesh forms the outer contour of a convex polygon.

[0042] Reference below Figure 2 and Figure 3 ,in Figure 2 is an exemplary schematic diagram of a BEV image with annotation information added according to one or more embodiments of the present application, Figure 3 FIG. 1 is an exemplary schematic diagram of a triangulation result according to one or more embodiments of the present application. Figure 2 As shown, curb marking lines and road internal restricted zone marking lines are added to the BEV image, wherein the curb marking lines are in the form of continuous lines, and the road internal restricted zone marking lines are mostly located in the area between the two curb marking lines. Figure 3 Any triangle in the triangulated network in satisfies the above empty circle property. It should be noted that the triangulation is only performed on the roadside annotation lines, so the generated triangulated network maintains the shape and features of the roadside in the original point set, which can serve as the basis for subsequent optimization (i.e., edge screening) so as to further improve the division quality of the drivable area through subsequent optimization.

[0043] In step S130, the edges constituting the triangular mesh are screened to obtain the outer contour of the drivable area that complies with the road structure rules.

[0044] like Figure 3 As shown, the triangulated network generated by triangulation includes a large number of redundant edges, which do not reflect the actual boundaries of the curb. In this regard, the present application proposes to screen the set of lines included in the triangulated network based on the road structure characteristics to retain the edges that comply with the road structure rules. It can be understood that the road structure rules are formulated based on the actual road structure characteristics (for example, intersections, road widths, and no-go zones). They are standards that describe, define, and constrain the actual road structure characteristics, and may cover the geometric dimensions, layout relationships, functional divisions, and mutual connections and transitions of the various components of the road (for example, lanes, curbs, intersections, no-go zones, etc.).

[0045] Optionally, in order to facilitate the screening of the edges of the triangle mesh, the edges constituting the triangle mesh can be divided into the following three categories based on the number of adjacent triangles: internal edges with an adjacent triangle number greater than 1, boundary edges with an adjacent triangle number of 1, and degenerate edges with an adjacent triangle number of 0. Several possible screening rules are listed below.

[0046] Rule 1: Eliminate all degenerate edges. It is understandable that degenerate edges have no adjacent triangles, which is usually caused by irregularities in the point set or data errors, and should therefore be eliminated.

[0047] Rule 2: Remove the boundary edges whose length is greater than the road width threshold. For example, the road width threshold is set according to the actual road width and the accuracy of manual annotation. Usually, the road width threshold can be set to 1 to 2.5 times the actual road width. The higher the accuracy of manual annotation, the lower the road width threshold.

[0048] Rule 3: Add boundary edges whose length is greater than the road width threshold to the filtering queue; select a boundary edge from the filtering queue and delete the selected boundary edge from the filtering queue; obtain the other two edges in the adjacent triangles of the selected boundary edge, reduce the number of adjacent triangles of the other two edges by one, and reclassify the other two edges based on the updated number of adjacent triangles; and update the filtering queue based on the reclassification result, and repeat the above operations for the updated filtering queue until the filtering queue is empty.

[0049] In a possible embodiment, the screening queue is first initialized, that is, the boundary edges e1, e2, and e3 whose lengths are greater than the road width threshold (for example, 1.5 times the actual road width) are added to the screening queue S, and S = {e1, e2, e3} at the initial time; it is assumed that e1 is selected first and deleted from S, and S = {e2, e3} at this time; the other two edges e4 and e3 of the only adjacent triangle T1 of e1 are obtained, where e4 is an internal edge with two adjacent triangles T1 and T2, and e3 is a boundary edge with only one adjacent triangle T1; then T1 is deleted from the adjacent triangles of e4 and e3, that is, the number of adjacent triangles of e4 and e3 is reduced by 1, and the number of adjacent triangles of e4 and e3 is updated to 1 and 0 respectively, and e4 and e3 are reclassified. After reclassification, e4 is a boundary edge and e3 is a degenerate edge; e3 is deleted from the screening queue S, and it is determined whether e4 is greater than the road width threshold. If e4 is greater than the road width threshold, S is updated to {e2, e4}; if e4 is less than or equal to the road width threshold, S is updated to {e2}; the above operation is repeated for the updated S until S is empty.

[0050] Rule 4: If the first endpoint of the first boundary edge belongs to the first curb marking line, the second endpoint belongs to the second curb marking line, and the first endpoint is not the line endpoint of the first curb marking line, then determine the line endpoint closest to the second endpoint on the first curb marking line; and move the first endpoint along the first curb marking line toward the determined line endpoint until the first endpoint moves to the determined line endpoint, or the first boundary edge intersects the restricted zone marking line inside the road, and update the moved first boundary edge to the new boundary edge.

[0051] Reference below Figure 4 ,in Figure 4 FIG. 1 is an exemplary schematic diagram of updating a boundary edge according to one or more embodiments of the present application. Figure 4 In the example, the annotation lines AB and CD are roadside annotation lines generated based on manual annotation information (generated via step S110), while the boundary line AE is an edge in the newly generated triangular mesh after triangulation (generated via step S120), that is, the boundary line AE does not belong to any roadside annotation line. Figure 4As shown, endpoint A of boundary line AE is the line endpoint of marked line AB, and endpoint E of boundary line AE is located inside marked line CD (non-line endpoint), so it meets the application conditions of rule 4. In this case, the lengths of AC and AD are calculated. In this embodiment, since AC is smaller than AD, endpoint E is moved toward C along marked line CD. If line segment AE does not contact the marked line of the restricted area inside the road during the movement (not shown), E moves to endpoint C and uses AC as the updated boundary line. If line segment AE begins to intersect with marked line l of the restricted area inside the road when E moves to E', as shown in FIG. Figure 4 As shown, AE' is taken as the updated boundary line.

[0052] Rule 5: If the second boundary edge is not included in any curb labeling line, and the other two edges in the adjacent triangle of the second boundary edge are included in the third curb labeling line, the second boundary edge is recorded as a degenerate edge, and the third curb labeling line is recorded as a boundary edge.

[0053] Reference below Figure 5 ,in Figure 5 FIG. 1 is an exemplary schematic diagram of updating a boundary edge according to one or more embodiments of the present application. Figure 5 As shown, the annotation line e is a roadside annotation line generated based on the manual annotation information (generated via step S110), where e includes Figure 5 The corner part in the boundary line FG is an edge in the newly generated triangular mesh after triangulation (generated via step S120), that is, the boundary line FG does not belong to any roadside annotation line. In addition, the corner part of the annotation line e and the boundary line FG form a triangle, thus meeting the application conditions of rule 5. At this time, the line segment FG is updated to a degenerate edge, and the entire annotation line e is recorded as a boundary edge.

[0054] Optionally, the application order of rules 2 and 3 is before that of rules 4 and 5, that is, the edges updated as new boundary edges according to rules 4 and 5 will not be removed even if their lengths are greater than the road width threshold.

[0055] Optionally, in order to facilitate the above-mentioned screening process, during the triangulation process, each curb annotation line can be marked and converted into a set of discrete points, and the points in the point set can be sorted to facilitate rapid search for the nearest point.

[0056] Optionally, all boundary edges retained after screening are recorded as valid boundary edges, and the outer contour of the drivable area is generated based on the valid boundary edges. Figure 6 is an exemplary schematic diagram of the outer contour of the drivable area according to one or more embodiments of the present application. Figure 2 The roadside marking lines in Figure 6The gaps in the outer contour of the drivable area of ​​the road are shown to have been completed to form a closed geometric pattern.

[0057] Furthermore, the road interior restricted zone marking line can also be closed. In one possible embodiment, if the distance between the two end points of the road interior restricted zone marking line is less than the distance threshold, the two end points can be connected by a straight line to form a closed area. In another possible embodiment, if the road interior restricted zone marking line is a U-shaped line, the two end points of the U-shaped line can be connected by a straight line. In other embodiments, the road interior restricted zone marking line can also be closed by smoothing the curve using a curve interpolation or fitting algorithm.

[0058] In step S140, based on the marking lines of the restricted area inside the road, an image segmentation method is used to divide the outer contour of the drivable area into a drivable area and a non-drivable area.

[0059] Optionally, in order to divide the drivable area and the non-drivable area inside the outer contour of the drivable area, and at the same time reduce the amount of calculation and improve the operation efficiency, the BEV image can be first gridded in step S140. The size of the grid can be determined according to the segmentation accuracy required for the final model output. The gridding step here reduces the complexity of the growth algorithm and improves the efficiency of automatic labeling.

[0060] Image segmentation is a key technology in digital image processing, which aims to divide a digital image into several different areas so that the pixels in each area have similar characteristics. The basic principle of image segmentation is to set a criterion function based on the grayscale, color, structure, texture and other characteristics of the image, and then set one or more thresholds. By comparing each pixel in the image with these thresholds, the image can be segmented into several non-overlapping areas. In one or more embodiments of the present application, after determining the outer contour of the drivable area, the area within the outer contour of the drivable area can be divided based on grayscale, color, structure, texture and other characteristics, and the drivable area and non-drivable area can be determined from the divided areas based on the current position or trajectory of the test vehicle.

[0061] Optionally, a region growing algorithm can be used to simultaneously implement the above-mentioned region division and region determination (that is, determine the drivable area and the non-drivable area from the divided area) to improve system efficiency. The region growing algorithm is an image segmentation technology based on pixel similarity. The core idea is to start from a seed point and gradually add neighboring pixels to the current area until a specific growth criterion or termination condition is met. Exemplarily, commonly used region growing algorithms include a four-connected domain growing algorithm and an eight-connected domain growing algorithm. The following will take the four-connected domain growing algorithm as an example to illustrate the application of the region growing algorithm.

[0062] Specifically, the grid where the test vehicle is currently located or where the trajectory is located can be selected as the origin (starting point) of the growth algorithm, and the default position of the vehicle is the drivable area. The outer contour of the drivable area or the marking line of the prohibited area inside the closed road is used as the growth stop condition, and the connected domain growth is performed to the adjacent grids. The growth criteria can be pre-defined to determine the similarity criteria between grids, for example, the difference in grayscale values ​​of adjacent grids is less than a certain threshold. During the growth process, a mark is initialized for all grids to indicate whether they have been included in a certain area. Starting from the starting point, check the adjacent grids in the four directions of up, down, left and right. If the adjacent grids meet the growth criteria and are not marked, add them to the drivable area and update their marking status. Repeat the above growth steps until there are no more grids inside the outer contour of the drivable area that can meet the growth conditions, or the growth stop condition is reached. At this time, the inner contour of the drivable area has been divided into several areas, including a drivable area and one or more non-drivable areas.

[0063] In step S150, drivable area labeling information is generated based on the division result. For example, the drivable area divided in the BEV image is labeled as a drivable area, and other areas (for example, one or more non-drivable areas divided in step S150, and the outside of the outer contour of the drivable area) are labeled as non-drivable areas. Figure 7 FIG. 1 is an exemplary schematic diagram of marking information of a drivable area according to one or more embodiments of the present application. Figure 7 As shown, the drivable area is marked in black and the non-drivable area is marked in white.

[0064] According to one or more embodiments of the present application, method 10 realizes the automatic labeling of the drivable area suitable for the field of autonomous driving. It can realize the construction of the outer contour of the drivable area that conforms to the road structure rules, and supports the automatic completion of missing road structures. At the same time, it has low requirements on the accuracy of manual labeling, and can still generate reliable drivable area labeling data even when the quality of manual labeling data is poor.

[0065] Figure 8 8 is a schematic block diagram of a drivable area marking device 80 according to one or more embodiments of the present application. The drivable area marking device 80 includes a memory 810, a processor 820, and a computer program 830 stored in the memory 810 and executable on the processor 820. The execution of the computer program 830 enables Figure 1 The illustrated drivable area marking method 10 is executed.

[0066] In addition, as described above, the present application can also be implemented as a computer storage medium, in which there is stored a program for causing a computer to execute the following steps: Figure 1Here, as the computer storage medium, various computer storage media such as disks (e.g., magnetic disks, optical disks, etc.), cards (e.g., memory cards, optical cards, etc.), semiconductor memories (e.g., ROMs, nonvolatile memories, etc.), and tapes (e.g., magnetic tapes, cassette tapes, etc.) can be used.

[0067] The present disclosure may also be implemented as a computer program product, which includes a computer program. When the computer program is executed by a processor, the following Figure 1 The procedure of method 10 is shown.

[0068] In the applicable situation, hardware, software or a combination of hardware and software can be used to realize the various embodiments provided by the application. Moreover, in the applicable situation, without departing from the scope of the application, the various hardware components and / or software components set forth herein can be combined into a composite component comprising software, hardware and / or both. In the applicable situation, without departing from the scope of the application, the various hardware components and / or software components set forth herein can be divided into subcomponents comprising software, hardware or both. In addition, in the applicable situation, it is contemplated that the software component can be implemented as a hardware component, and vice versa.

[0069] Software (such as program code and / or data) according to the present application can be stored on one or more computer storage media. It is also contemplated that the software identified herein can be implemented using one or more general or special-purpose computers and / or computer systems, networked and / or otherwise. Where applicable, the order of the various steps described herein can be changed, combined into composite steps, and / or divided into sub-steps to provide the features described herein.

[0070] The embodiments and examples set forth herein are provided to best illustrate embodiments according to the present application and its specific applications, and thereby enable those skilled in the art to implement and use the present application. However, those skilled in the art will appreciate that the above description and examples are provided only for ease of illustration and example. The description set forth is not intended to cover all aspects of the present application or to limit the present application to the precise form disclosed.

Claims

1. A method for marking a drivable area, characterized in that: The method comprises the following steps: Generate roadside marking lines and road internal restricted zone marking lines on the BEV image based on manual marking information; triangulate the point set constituting the roadside marking line using a triangulation algorithm to obtain a triangular mesh; Screening the edges constituting the triangular mesh to obtain an outer contour of a drivable area that complies with road structure rules; Based on the marked lines of the restricted area inside the road, dividing the outer contour of the drivable area into a drivable area and a non-drivable area by using an image segmentation method; and Generate drivable area annotation information based on the division results.

2. The method for marking a drivable area according to claim 1, characterized in that: Screening the edges constituting the triangular mesh includes: Dividing the edges constituting the triangular mesh into the following three categories based on the number of adjacent triangles: internal edges with an adjacent triangle number greater than 1, boundary edges with an adjacent triangle number of 1, and degenerate edges with an adjacent triangle number of 0; and Eliminate border edges whose length is greater than the road width threshold.

3. The method for marking a drivable area according to claim 2, characterized in that: Screening the edges constituting the triangular mesh further includes: Adding the boundary edges whose length is greater than the road width threshold to the screening queue; Selecting a boundary edge from the screening queue and deleting the selected boundary edge from the screening queue; Obtaining the other two edges in the adjacent triangles of the selected boundary edge, reducing the number of adjacent triangles of the other two edges by one, and reclassifying the other two edges based on the updated number of adjacent triangles; and The screening queue is updated based on the reclassification result, and the above operation is repeated for the updated screening queue until the screening queue is empty.

4. The method for marking a drivable area according to claim 3, characterized in that: Screening the edges constituting the triangular mesh based on the road structure rule also includes: If the first endpoint of the first boundary edge belongs to the first curb labeling line, the second endpoint belongs to the second curb labeling line, and the first endpoint is not a line endpoint of the first curb labeling line, then determining the line endpoint closest to the second endpoint on the first curb labeling line; Moving the first endpoint along the first roadside marking line toward the determined line endpoint until the first endpoint moves to the determined line endpoint or the first boundary edge intersects the road interior restricted zone marking line; and Update the moved first border edge to the new border edge.

5. The method for marking a drivable area according to claim 2, characterized in that: Screening the edges constituting the triangular mesh further includes: If the second boundary edge is not included in any curb marking line, and the other two edges in the adjacent triangle of the second boundary edge are included in the third curb marking line, the second boundary edge is recorded as a degenerate edge, and the third curb marking line is recorded as a boundary edge.

6. The method for marking a drivable area according to any one of claims 3 to 5, characterized in that: The edges constituting the triangular mesh are screened to obtain the outer contour of the drivable area that complies with the road structure rules, including: Record the boundary edges retained after screening as valid boundary edges; and The outer contour of the drivable area is generated based on the effective boundary edge.

7. The method for marking a drivable area according to claim 1, characterized in that: The method further comprises: The marking line of the restricted zone inside the road is closed, wherein the marking line of the restricted zone inside the road is closed including: If the distance between the two endpoints of the restricted zone marking line inside the road is less than the distance threshold, the two endpoints are connected by a straight line; and / or If the restricted area marking line inside the road is a U-shaped line, two end points of the U-shaped line are connected by a straight line.

8. The method for marking a drivable area according to claim 1, characterized in that: The image segmentation method is a region growing algorithm, and Based on the marked line of the restricted area inside the road, dividing the drivable area and the non-drivable area within the outer contour of the drivable area by using an image segmentation method includes: Performing gridding processing on the BEV image; The outer contour of the drivable area and the marking line of the prohibited area inside the closed road are used as growth stop conditions, and the grid where the vehicle is located is used as a starting point to grow toward the adjacent grids.

9. A drivable area marking device, characterized in that: The method comprises: a memory; a processor; and a computer program stored in the memory and executable on the processor, wherein the execution of the computer program enables the drivable area labeling method according to any one of claims 1 to 8 to be executed.

10. A computer storage medium, characterized in that: The computer storage medium includes instructions, which, when executed, execute the drivable area marking method according to any one of claims 1-8.

Images