Lane line instance detection method and device, equipment and storage medium

Abstract

The invention provides a lane line instance detection method and device, equipment and a storage medium. The method comprises the following steps: generating a lane line mask through a segmentation network; traversing the lane line mask from bottom to top to obtain all lane line solid line segments; determining the direction of the solid line segment of each lane line; all the lane line solid line segments are determined as processing solid line segments; determining a fitting curve, a starting point coordinate and an end point coordinate of each processed real line segment; determining a cost matrix according to the trend, the starting point coordinate and the end point coordinate of each processed real line segment; and obtaining a lane line instance according to the fitted curve and the cost matrix. According to the method provided by the invention, the real line segment of the lane line is obtained, the cost matrix is formed according to the starting point coordinates and the end point coordinates of the real line segment, the lane line instance is obtained according to the fitted curve and the cost matrix, and the accurate lane line instance can be obtained without depending on other algorithms.

Description

Technical Field

[0001] The present application relates to the field of autonomous driving technology, and in particular to a lane line instance detection method, device, equipment, and storage medium. Background Art

[0002] Lane line detection plays an important role in autonomous driving, but the self-driving system's demand for lane lines is no longer a simple segmentation task. In addition to outputting which pixel locations are lane lines, it is also necessary to output which pixels correspond to the same lane line, the so-called lane line instance.

[0003] An existing lane line instance detection method is mainly a key point based method. This type of method directly detects lane line instances and uses post-processing to divide the instances. This method may have challenges in the accuracy of key point detection and is highly dependent on post-processing algorithms. Summary of the invention

[0004] In order to solve one of the above-mentioned technical defects, the present application provides a lane line instance detection method, device, equipment, and storage medium.

[0005] In a first aspect of the present application, a lane line instance detection method is provided, the method comprising: Generate lane mask through segmentation network; Traverse the lane line mask from bottom to top and obtain all the solid lane line segments; Determine the direction of the solid line segment of each lane line; wherein the direction is horizontal or vertical; All lane line solid segments are determined as processed solid line segments; Determine the fitting curve, starting point coordinates and end point coordinates of each processed solid line segment; Determine the cost matrix according to the direction, starting point coordinates and end point coordinates of each processed solid line segment; Get the lane line instance based on the fitting curve and cost matrix.

[0006] Optionally, a cost matrix is ​​determined according to the direction, starting point coordinates and end point coordinates of each processed solid line segment, including: According to the starting point coordinates and the end point coordinates, the direction coordinate set of each processed solid line segment is determined; if the direction of the processed solid line segment is horizontal, the direction coordinate set is a set consisting of its horizontal coordinates; if the direction of the processed solid line segment is vertical, the direction coordinate set is a set consisting of its vertical coordinates; Determine the cost values ​​of any two processed solid line segments respectively; if the directions of any two processed solid line segments are the same and the intersection of the direction coordinate sets is not empty, or if the directions of any two processed solid line segments are different, then the cost values ​​of any two processed solid line segments are positive infinity; otherwise, the cost value is non-positive infinity; All cost values ​​are formed into a cost matrix; where the cost matrix is matrix, The total number of solid line segments processed.

[0007] Optionally, a lane line instance is obtained according to the fitting curve and the cost matrix, including: Adjust the cost matrix according to the fitting curve, starting point coordinates and end point coordinates of each processed solid line segment; The processed solid line segments are numbered from small to large according to the ordinate values ​​of the starting coordinates; The processed solid line segments are merged according to the numbers and the adjusted cost matrix to obtain lane line instances.

[0008] Optionally, the cost matrix is ​​adjusted according to the fitting curve, the starting point coordinates and the end point coordinates of each processed solid line segment, including: Determine the non-positive infinite cost value in the cost matrix as the cost value to be adjusted; According to the fitting curve, the starting point coordinates and the end point coordinates, the fitting error between the two processed solid line segments involved in each cost value to be adjusted is determined. and distance error ; according to and fitting error threshold The relationship between Distance error threshold The relationship between them is used to adjust the cost value to be adjusted.

[0009] Optionally, the fitting error between the two processed solid line segments involved in each cost value to be adjusted is determined according to the fitting curve, the starting point coordinates and the end point coordinates. and distance error ,include: Calculate the first error between the starting point of the upper line segment and the fitting curve of the lower line segment according to the starting point coordinates of the upper line segment; wherein the upper line segment is one of the two lane line solid line segments involved, the lower line segment is the other of the two lane line solid line segments involved, and the upper line segment is located above the lower segment; Calculate the second error between the end point of the lower line segment and the fitting curve of the upper line segment according to the coordinates of the end point of the lower line segment; Sure = first error + second error; According to the coordinates of the starting point of the upper line segment and the coordinates of the end point of the lower line segment, the distance between the starting point of the upper line segment and the end point of the lower line segment is determined and determined as .

[0010] Optionally, according to and fitting error threshold The relationship between Distance error threshold The relationship between , the cost value to be adjusted is adjusted, including: for ,or, The cost value to be adjusted is adjusted to positive infinity; for ,and, The cost value to be adjusted is adjusted to ,in, is the weight coefficient of the fitting error.

[0011] Optionally, the processed solid line segments are merged according to the numbers and the adjusted cost matrix to obtain lane line instances, including: All treated solid line segments are marked as untreated; Select one processed solid line segment marked as unprocessed in order from the smallest to the largest number; if it is determined based on the adjusted cost matrix that there is a non-infinite cost value between the selected processed solid line segment and other processed solid line segments marked as unprocessed, then merge the selected processed solid line segment with the other processed solid line segments marked as unprocessed with the smallest cost value to obtain a merged processed solid line segment; mark the selected processed solid line segment and other processed solid line segments marked as unprocessed with the smallest cost value as processed; The processed solid line segments are updated to all processed solid line segments marked as unprocessed and merged processed solid line segments, and the steps of determining the fitting curve, starting point coordinates and end point coordinates of each processed solid line segment and subsequent steps are repeated until there are no new merged processed solid line segments.

[0012] In a second aspect of the present application, a lane line instance detection device is provided, the device comprising: The generation module is used to generate lane mask through the segmentation network; The acquisition module is used to traverse the lane line mask generated by the generation module from bottom to top and obtain all the lane line solid segments; A first determination module is used to determine the direction of the solid line segments of each lane line obtained by the acquisition module; wherein the direction is horizontal or vertical; A second determination module is used to determine all lane line solid line segments acquired by the acquisition module as processed solid line segments; A third determination module is used to determine the fitting curve, the starting point coordinates and the end point coordinates of each processed solid line segment determined by the second determination module; A fourth determination module, used to determine a cost matrix according to the direction of each processed solid line segment determined by the first determination module and the starting point coordinates and the end point coordinates determined by the third determination module; The fifth determination module is used to obtain a lane line instance according to the fitting curve determined by the third determination module and the cost matrix determined by the fourth determination module.

[0013] In a third aspect of the present application, an electronic device is provided, including: Memory; Processor; and Computer programs; The computer program is stored in the memory and is configured to be executed by the processor to implement the method as described in the first aspect above.

[0014] In a fourth aspect of the present application, a computer-readable storage medium is provided, on which a computer program is stored; the computer program is executed by a processor to implement the method described in the first aspect above.

[0015] The present application provides a lane line instance detection method, device, equipment, and storage medium, the method comprising: generating a lane line mask through a segmentation network; traversing the lane line mask from bottom to top to obtain all lane line solid line segments; determining the direction of each lane line solid line segment; determining all lane line solid line segments as processed solid line segments; determining the fitting curve, starting point coordinates, and end point coordinates of each processed solid line segment; determining a cost matrix based on the direction, starting point coordinates, and end point coordinates of each processed solid line segment; and obtaining a lane line instance based on the fitting curve and the cost matrix. The method provided in the present application obtains a lane line solid line segment, forms a cost matrix based on its starting point coordinates and end point coordinates, obtains a lane line instance based on the fitting curve and the cost matrix, and can obtain an accurate lane line instance without relying on other algorithms. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings: Figure 1 A schematic diagram of a process flow of a lane line instance detection method provided in an embodiment of the present application; Figure 2 A schematic diagram of processing a solid line segment provided in an embodiment of the present application; Figure 3A schematic diagram of two processed solid line segments with the same direction and overlap provided in an embodiment of the present application; Figure 4 A schematic diagram of the structure of a lane line instance detection device provided in an embodiment of the present application; Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0017] In order to make the technical solutions and advantages in the embodiments of the present application more clearly understood, the exemplary embodiments of the present application are further described in detail below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than an exhaustive list of all the embodiments. It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict.

[0018] In the process of realizing this application, the inventors found that lane line detection plays an important role in autonomous driving, but the demand of the self-driving system for lane lines is no longer a simple segmentation task. In addition to outputting which pixel positions are lane lines, it is also necessary to output which pixels correspond to the same lane line, namely the so-called lane line instances. An existing lane line instance detection method is mainly based on key point methods. This type of method directly detects instances of lane lines and uses post-processing to divide the instances. This method may have challenges in the accuracy of key point detection and has a high dependence on post-processing algorithms.

[0019] In response to the above problems, the embodiments of the present application provide a lane line instance detection method, device, equipment, and storage medium, the method comprising: generating a lane line mask through a segmentation network; traversing the lane line mask from bottom to top to obtain all lane line solid line segments; determining the direction of each lane line solid line segment; determining all lane line solid line segments as processed solid line segments; determining the fitting curve, starting point coordinates, and end point coordinates of each processed solid line segment; determining the cost matrix according to the direction, starting point coordinates, and end point coordinates of each processed solid line segment; and obtaining a lane line instance according to the fitting curve and the cost matrix. The method provided in the present application obtains a lane line solid line segment, forms a cost matrix according to its starting point coordinates and end point coordinates, obtains a lane line instance according to the fitting curve and the cost matrix, and can obtain an accurate lane line instance without relying on other algorithms.

[0020] See also Figure 1 The embodiment provides a lane line instance detection method, and the implementation process of the method is as follows: 101, generate lane mask through segmentation network.

[0021] 102, traverse the lane line mask from bottom to top to obtain all the lane line solid segments.

[0022] Among them, any solid lane line segment is a polygon composed of all the pixels on the lane line. Therefore, the solid lane line segment in this step does not have width in only one dimension (that is, it is not a line, such as width only in the horizontal direction, or width only in the vertical direction), but has width in both the horizontal and vertical directions (that is, it is a polygon), and all the pixels in the polygon belong to the same lane line.

[0023] 103, determine the direction of the solid line segment of each lane line.

[0024] Here, the direction is horizontal or vertical.

[0025] Since any lane line solid line segment has width in both the horizontal and vertical directions, the direction of the lane line solid line segment can be obtained according to the relationship between the widths in the horizontal and vertical directions. For example, if the horizontal width of any lane line solid line segment is greater than the vertical width, then the lane line solid line segment is horizontal, otherwise, the lane line solid line segment is vertical.

[0026] 104, all lane line solid line segments are determined as processed solid line segments.

[0027] 105, determining the fitting curve, the starting point coordinates and the end point coordinates of each processed solid line segment.

[0028] For example, for any processed solid line segment, the curve formed by its center point is fitted to obtain its cubic fitting curve (the cubic fitting curve is the fitting curve of any processed solid line segment) and its starting point coordinates and end point coordinates.

[0029] Because any solid line segment of a lane line has width in both the horizontal and vertical directions, the starting point coordinates and the end point coordinates are a set, that is, the starting point coordinates of any processed solid line segment are the coordinate set of all pixel points located at the starting point of any lane line solid line segment, and the end point coordinates are the coordinate set of all pixel points located at the end point of any lane line solid line segment.

[0030] 106. Determine a cost matrix according to the direction, starting point coordinates and end point coordinates of each processed solid line segment.

[0031] The implementation process of this step is: 106-1, determining the direction coordinate set of each processed solid line segment according to the starting point coordinates and the end point coordinates.

[0032] If the direction of the solid line segment is horizontal, the direction coordinate set is the set consisting of its horizontal coordinates; if the direction of the solid line segment is vertical, the direction coordinate set is the set consisting of its vertical coordinates.

[0033] For example, for any processing solid line segment In step 106-1, the processed solid line segment is obtained. The coordinates of all the points covered, if it is determined in step 103 to process the solid line segment If the direction is horizontal, the horizontal coordinates (i.e. x-axis coordinates) of all points are summed up to form a solid line segment. If it is determined in step 103 that a solid line segment is processed If the direction is vertical, the ordinates (i.e. y-axis coordinates) of all points are summed up to form a solid line segment. The set of direction coordinates.

[0034] 106-2, respectively determine the cost values ​​of any two processed solid line segments.

[0035] Among them, if the directions of any two processed solid line segments are the same and the intersection of the direction coordinate sets is not empty, or if the directions of any two processed solid line segments are different, then the cost value of any two processed solid line segments is positive infinity; otherwise, the cost value is not positive infinity.

[0036] The processed solid line segments in step 106-2 are randomly obtained from all processed solid line segments. To process solid line segments, first Choose one of the processed solid line segments, and then Choose one of the processed solid line segments, so the combination of any two processed solid line segments has , then the cost value also has indivual.

[0037] Take any processed solid line segment as the processed solid line segment , any other processing solid line segment is a processing solid line segment For example, processing solid line segments Can handle solid line segments For the same processing of solid line segments, you can also process solid line segments with For different processing of solid line segments. The implementation process of step 106-2 is as follows: 1. Get and process the solid line segment Direction 1, processing solid line segments Direction 2. Get the processing solid line segment The direction coordinate set 1, processing the solid line segment The direction coordinate set 2.

[0038] 2. Determine whether trend 1 and trend 2 are the same, and determine whether the intersection of trend coordinate set 1 and trend coordinate set 2 is empty.

[0039] If we process solid line segments and processing solid line segments The same direction and overlapping parts (such as Figure 3 1 processes solid line segments and 3 processes solid line segments), then the intersection of the direction coordinate set 1 and the direction coordinate set is not empty. and processing solid line segments If the directions are the same but there is no overlap, the intersection of ordinate set 1 and ordinate set 2 is empty.

[0040] 3. If direction 1 and direction 2 are the same, and the intersection of ordinate set 1 and ordinate set 2 is not empty, then process the solid line segment Can handle solid line segments The corresponding cost value is positive infinity.

[0041] If we process solid line segments Can handle solid line segments For the same processed solid line segment, direction 1 and direction 2 must be the same, and the intersection of ordinate set 1 and ordinate set 2 must be non-empty, then its cost value is positive infinity. In other words, the cost value of a processed solid line segment and itself is positive infinity.

[0042] If we process solid line segments Can handle solid line segments If they are not the same solid line segment, their directions can be the same or different. If direction 1 and direction 2 are the same, and the intersection of ordinate set 1 and ordinate set 2 is not empty (such as Figure 2 In other words, the cost value of a processed solid line segment and other processed solid line segments with the same direction and overlap is positive infinity.

[0043] 4. If trends 1 and 2 are different (e.g. Figure 2 In the above example, if number 1 processes the solid line segment and number 6 processes the solid line segment), then its cost value is positive infinity.

[0044] 5. Otherwise (that is, direction 1 and direction 2 are the same, but the intersection of ordinate set 1 and ordinate set 2 is empty), the cost value is not positive infinity.

[0045] Therefore, the cost value represents the distance between the two lane line solid line segments corresponding to it. If the directions of any two processed solid line segments are the same and the intersection of the direction coordinate set is not empty, it means that the two processed solid line segments overlap, and their distance is positive infinity. If the directions of any two processed solid line segments are different, it means that the two processed solid line segments are two processed solid line segments in vertical directions, so their distance is also positive infinity. If the directions of any two processed solid line segments are the same but the intersection of the direction coordinate set is empty, it means that the two processed solid line segments are in the same direction, and there is no overlap, and there is a certain distance, so the cost value is non-positive infinity. In specific implementation, non-positive infinity can be a preset fixed value, or it can be the actual distance between the two processed solid line segments, that is, the distance between the starting point of the upper processed solid line segment and the end point of the lower processed solid line segment in the two processed solid line segments.

[0046] 106-3, all cost values ​​are formed into a cost matrix.

[0047] Among them, the cost matrix is matrix, The total number of solid line segments processed.

[0048] like Figure 2 The cost matrix of the processed solid line segment shown in Table 1 is shown in Table 1, where NP represents non-positive infinity (it does not limit its value, nor does it limit whether the specific values ​​of each NP are the same, it only indicates that the value here is non-positive infinity), and ∞ represents positive infinity.

[0049] Table 1

[0050] 107, a lane line instance is obtained according to the fitting curve and the cost matrix.

[0051] The implementation process of this step is: 107-1, adjusting the cost matrix according to the fitting curve, the starting point coordinates and the end point coordinates of each processed solid line segment.

[0052] The process of adjusting the cost matrix is: 1. Determine the non-positive infinite cost values ​​in the cost matrix as the cost values ​​to be adjusted.

[0053] For example, the NPs in Table 1 are all cost values ​​to be adjusted.

[0054] It should be noted that NP only means that its value is positive infinity, and does not represent a specific value.

[0055] 2. According to the fitting curve, the starting point coordinates and the end point coordinates, determine the fitting error between the two processed solid line segments involved in each cost value to be adjusted and distance error .

[0056] For example, determine and The determination process is: 1) According to the coordinates of the starting point of the upper line segment, calculate the first error between the starting point of the upper line segment and the fitting curve of the lower line segment.

[0057] Among them, the upper line segment is one of the two lane line solid line segments involved, the lower line segment is the other of the two lane line solid line segments involved, and the upper line segment is located above the lower segment.

[0058] by Figure 2 For example, the upper segment is the processed solid line segment 4, and the lower segment is the processed solid line segment 1. As can be seen from the description of step 105, the starting point coordinates of the upper segment are a coordinate set (such as set4) consisting of the coordinates of all pixels located at the starting point of the processed solid line segment 4. Then the first error between the starting point of the processed solid line segment 4 and the fitting curve of the processed solid line segment 1 (such as curve1) is the fitting error of set4 on curve1.

[0059] For example, the expression of curve1 is ,in, , , is the coefficient, is a constant. If a point in set4 , whose coordinates are , then point The fitting error on curve1 is If there is The coordinates of the points, then the first error is .

[0060] 2) According to the coordinates of the end point of the lower line segment, calculate the second error between the end point of the lower line segment and the fitting curve of the upper line segment.

[0061] Still Figure 2 For example, the upper segment is the processed solid line segment 4, and the lower segment is the processed solid line segment 1. As can be seen from the description of step 105, the end point coordinates of the lower segment are a coordinate set (such as set1) consisting of the coordinates of all pixel points located at the end point of the processed solid line segment 1. Then the second error between the end point of the processed solid line segment 1 and the fitting curve of the processed solid line segment 4 (such as curve4) is the fitting error of set1 on curve4.

[0062] For example, the expression of curve4 is ,in, , , is the coefficient, is a constant. If a point in set1 , whose coordinates are , then point The fitting error on curve4 is If there is The coordinates of the points, then the first error is .

[0063] 3) Determine =first error + second error.

[0064] 4) According to the coordinates of the starting point of the upper line segment and the end point of the lower line segment, determine the distance between the starting point of the upper line segment and the end point of the lower line segment and determine it as .

[0065] The distance here can be Euclidean distance.

[0066] Still Figure 2 For example, the upper segment is the processed solid line segment 4, and the lower segment is the processed solid line segment 1. As can be seen from the description of step 105, the starting coordinates of the upper segment are a coordinate set consisting of the coordinates of all pixels located at the starting point of the processed solid line segment 4 (such as set4), and the end coordinates of the lower segment are a coordinate set consisting of the coordinates of all pixels located at the end point of the processed solid line segment 1 (such as set1). Then is the Euclidean distance between set4 and set1.

[0067] 3. According to and fitting error threshold The relationship between Distance error threshold The relationship between them is used to adjust the cost value to be adjusted.

[0068] For example, for ,or, The cost value to be adjusted is adjusted to positive infinity. ,and, The cost value to be adjusted is adjusted to ,in, is the weight coefficient of the fitting error.

[0069] Still Figure 2 For example, if No. 1 and No. 4 in the processing of solid line segments are used, ,or, , then adjust it to positive infinity. That is, we get the adjusted cost matrix shown in Table 2: Table 2

[0070] like ,and, , then adjust it to That is, the adjusted cost matrix shown in Table 3 is obtained: Table 3

[0071] 107-2, the processed solid line segments are numbered from small to large according to the ordinate values ​​of the starting coordinates.

[0072] As can be seen from the description of step 105, the starting point coordinates are a coordinate set consisting of the coordinates of all pixel points at the starting point of the processed solid line segment. Therefore, in the specific implementation of step 107-2, the minimum value of the ordinate in the set of starting coordinates of each processed solid line segment can be determined, and the processed solid line segments can be numbered according to the minimum value. If the minimum values ​​of the ordinates of at least two processed solid line segments are the same, the numbers between them can be randomly determined, or they can be sorted according to the minimum values ​​of the abscissas. For example, for two processed solid line segments with the same minimum ordinate values, the minimum value of the abscissa is numbered before the other, for example, numbering as follows: Figure 2 shown.

[0073] 107-3, the processed solid line segments are merged according to the numbers and the adjusted cost matrix to obtain lane line instances.

[0074] This step will merge based on the distance between the solid line segments and the similarity of the fitted curves. The specific implementation process is as follows: 1. Mark all processed solid line segments as unprocessed.

[0075] 2. Select a processed solid line segment marked as unprocessed in order from the smallest to the largest number. If, based on the adjusted cost matrix, it is determined that there is a non-infinite cost value between the selected processed solid line segment and other processed solid line segments marked as unprocessed, then merge the selected processed solid line segment with the other processed solid line segments marked as unprocessed with the smallest cost value to obtain a merged processed solid line segment. Mark the selected processed solid line segment and other processed solid line segments marked as unprocessed with the smallest cost value as processed.

[0076] In this step, the processed solid line segments marked as unprocessed and with the smallest number (such as processed solid line segments) are calculated in turn according to the numbers of the processed solid line segments. ) and other processed solid line segments marked as unprocessed. If there is a non-infinite cost value, determine the unprocessed processed solid line segment with the smallest cost value (such as the processed solid line segment ), solid line segments will be processed and processing solid line segments Merge into one (such as processing solid line segments ), according to the processing of solid line segments Fitting curve, starting point coordinates, end point coordinates and processing solid line segments The fitting curve, starting point coordinates and end point coordinates are processed into a solid line segment The fitting curve, starting point coordinates and end point coordinates of the solid line segment. Fitting curves and processing solid line segments The fitting curve is fitted to obtain the processed solid line segment The solid line segment will be processed and processing solid line segments The end point coordinates of the upper line segment in the processing are used as the solid line segment The end point coordinates of the solid line segment will be processed and processing solid line segments The starting point coordinates of the lower line segment in the processing solid line segment The starting point coordinates of . Processing solid line segments Direction and processing of solid line segments and processing solid line segments Same (according to the description of the cost matrix determination process in step 106, only the cost values ​​of the two processed solid line segments with the same direction will be non-positive infinity, so the processed solid line segments and processing solid line segments The trend must be the same).

[0077] If there is no non-infinite cost value, it means that there are no other processed solid line segments that need to be merged with the processed solid line segment, and no processing is performed.

[0078] Still Figure 2Take as an example, if the adjusted cost matrix is ​​as shown in Table 1, then for the processed solid line segment No. 1, compare its cost value with that of the processed solid line segment No. 2, its cost value with that of the processed solid line segment No. 3, its cost value with that of the processed solid line segment No. 4, its cost value with that of the processed solid line segment No. 5, and its cost value with that of the processed solid line segment No. 6. If the non-positive infinite and minimum cost value is the cost value of the processed solid line segment No. 4, then merge the processed solid line segment No. 1 and the processed solid line segment No. 4 into a new processed solid line segment, and obtain the fitting curve, starting point coordinates, and end point coordinates of the new processed solid line segment. Mark the processed solid line segment No. 1 and the processed solid line segment No. 4 as processed. Then process the processed solid line segment No. 2, etc., until all processed solid line segments are selected.

[0079] 3. Update the processed solid line segments to all processed solid line segments marked as unprocessed and merged processed solid line segments, and repeat the step of determining the fitting curve, starting point coordinates and end point coordinates of each processed solid line segment (i.e., step 105) and subsequent steps until there are no new merged processed solid line segments.

[0080] That is, if there are merged processed solid line segments in the above 107-3 2-merging process, then the merged processed solid line segments and all the processed solid line segments marked as unprocessed will be used as the processed solid line segments to be processed next time, and step 105 and subsequent steps will be re-executed. If there are no new merged processed solid line segments in the above 107-3 2-merging process, then all lane line solid line segments belonging to a lane line instance are merged to obtain a lane line instance, thereby completing the execution of the lane line instance detection method provided in this embodiment, and all the processed solid line segments at this time (that is, all the processed solid line segments marked as unprocessed) are used as the final detected lane line instance.

[0081] This embodiment provides a lane line instance detection method, which generates a lane line mask through a segmentation network; traverses the lane line mask from bottom to top to obtain all lane line solid line segments; determines the direction of each lane line solid line segment; determines all lane line solid line segments as processed solid line segments; determines the fitting curve, starting point coordinates and end point coordinates of each processed solid line segment; determines the cost matrix according to the direction, starting point coordinates and end point coordinates of each processed solid line segment; obtains the lane line instance according to the fitting curve and the cost matrix. The method of this embodiment obtains the lane line solid line segment, forms a cost matrix according to its starting point coordinates and end point coordinates, obtains the lane line instance according to the fitting curve and the cost matrix, and can obtain an accurate lane line instance without relying on other algorithms.

[0082] Based on the same inventive concept of the lane line instance detection method, this embodiment provides a lane line instance detection device, such as Figure 4 As shown, the device comprises: A generation module 401, used to generate a lane mask through a segmentation network; An acquisition module 402 is used to traverse the lane line mask generated by the generation module 401 from bottom to top to obtain all lane line solid line segments; The first determination module 403 is used to determine the direction of the solid line segments of each lane line obtained by the acquisition module 402; wherein the direction is horizontal or vertical; A second determination module 404 is used to determine all lane line solid line segments acquired by the acquisition module 402 as processed solid line segments; The third determination module 405 is used to determine the fitting curve, the starting point coordinates and the end point coordinates of each processed solid line segment determined by the second determination module 404; The fourth determination module 406 is used to determine the cost matrix according to the direction of each processed solid line segment determined by the first determination module 403 and the starting point coordinates and the end point coordinates determined by the third determination module 405; The fifth determination module 407 is used to obtain a lane line instance according to the fitting curve determined by the third determination module 405 and the cost matrix determined by the fourth determination module 406 .

[0083] Among them, the fourth determination module 406 is used to determine the direction coordinate set of each processed solid line segment according to the starting point coordinates and the end point coordinates. Among them, if the direction of the processed solid line segment is horizontal, the direction coordinate set is a set consisting of its horizontal coordinates. If the direction of the processed solid line segment is vertical, the direction coordinate set is a set consisting of its vertical coordinates. Determine the cost values ​​of any two processed solid line segments respectively. Among them, if the directions of any two processed solid line segments are the same and the intersection of the direction coordinate set is not empty, or if the directions of any two processed solid line segments are different, then the cost value of any two processed solid line segments is positive infinity. Otherwise, the cost value is not positive infinity. All cost values ​​are formed into a cost matrix. Among them, the cost matrix is matrix, The total number of solid line segments processed.

[0084] The fifth determination module 407 is used to adjust the cost matrix according to the fitting curve, the starting point coordinates and the end point coordinates of each processed solid line segment. The processed solid line segments are numbered from small to large according to the ordinate values ​​of the starting coordinates. The processed solid line segments are merged according to the numbering and the adjusted cost matrix to obtain a lane line instance.

[0085] Among them, according to the fitting curve, the starting point coordinates and the end point coordinates of each processed solid line segment, the cost matrix is ​​adjusted, including: The non-positive infinite cost values ​​in the cost matrix are determined as the cost values ​​to be adjusted.

[0086] According to the fitting curve, the starting point coordinates and the end point coordinates, the fitting error between the two processed solid line segments involved in each cost value to be adjusted is determined. and distance error .

[0087] according to and fitting error threshold The relationship between Distance error threshold The relationship between them is used to adjust the cost value to be adjusted.

[0088] Among them, according to the fitting curve, the starting point coordinates and the end point coordinates, the fitting error between the two processed solid line segments involved in each cost value to be adjusted is determined and distance error ,include: According to the starting point coordinates of the upper line segment, a first error between the starting point of the upper line segment and the fitting curve of the lower line segment is calculated, wherein the upper line segment is one of the two lane line solid line segments involved, the lower line segment is the other of the two lane line solid line segments involved, and the upper line segment is located above the lower segment.

[0089] According to the coordinates of the end point of the lower line segment, a second error between the end point of the lower line segment and the fitting curve of the upper line segment is calculated.

[0090] Sure =first error + second error.

[0091] According to the coordinates of the starting point of the upper line segment and the coordinates of the end point of the lower line segment, the distance between the starting point of the upper line segment and the end point of the lower line segment is determined and determined as .

[0092] Among them, according to and fitting error threshold The relationship between Distance error threshold The relationship between , the cost value to be adjusted is adjusted, including: for ,or, The cost value to be adjusted is adjusted to positive infinity.

[0093] for ,and, The cost value to be adjusted is adjusted to ,in, is the weight coefficient of the fitting error.

[0094] The processed solid line segments are merged according to the number and the adjusted cost matrix to obtain lane line instances, including: All treated solid line segments are marked as untreated.

[0095] Select a processed solid line segment marked as unprocessed in order from the smallest to the largest number. If, based on the adjusted cost matrix, it is determined that there is a non-infinite cost value between the selected processed solid line segment and other processed solid line segments marked as unprocessed, then merge the selected processed solid line segment with the other processed solid line segments marked as unprocessed with the smallest cost value to obtain a merged processed solid line segment. Mark the selected processed solid line segment and other processed solid line segments marked as unprocessed with the smallest cost value as processed.

[0096] The processed solid line segments are updated to all processed solid line segments marked as unprocessed and merged processed solid line segments, and the steps of determining the fitting curve, starting point coordinates and end point coordinates of each processed solid line segment and subsequent steps are repeated until there are no new merged processed solid line segments.

[0097] The device provided in this embodiment, after obtaining a solid lane line segment, forms a cost matrix according to its starting point coordinates and end point coordinates, obtains a lane line instance according to the fitting curve and the cost matrix, and can obtain an accurate lane line instance without relying on other algorithms.

[0098] Based on the same inventive concept of the lane line instance detection method, this embodiment provides an electronic device, such as Figure 5 As shown, it includes: a memory 501, a processor 502, and a computer program.

[0099] The computer program is stored in the memory 501 and is configured to be executed by the processor 502 to implement the above lane line instance detection method.

[0100] Specifically, Generate lane mask through segmentation network.

[0101] Traverse the lane line mask from bottom to top and obtain all the solid lane line segments.

[0102] Determine the direction of the solid line segment of each lane line, where the direction is horizontal or vertical.

[0103] All lane line solid line segments are determined as processed solid line segments.

[0104] Determine the fitting curve, starting point coordinates and end point coordinates of each processed solid line segment.

[0105] The cost matrix is ​​determined according to the direction, starting point coordinates and end point coordinates of each processed solid line segment.

[0106] Get the lane line instance based on the fitting curve and cost matrix.

[0107] Optionally, a cost matrix is ​​determined according to the direction, starting point coordinates and end point coordinates of each processed solid line segment, including: According to the starting point coordinates and the end point coordinates, the direction coordinate set of each processed solid line segment is determined. If the direction of the processed solid line segment is horizontal, the direction coordinate set is the set consisting of its horizontal coordinates. If the direction of the processed solid line segment is vertical, the direction coordinate set is the set consisting of its vertical coordinates.

[0108] Determine the cost value of any two processed solid line segments respectively. If the directions of any two processed solid line segments are the same and the intersection of the direction coordinate sets is not empty, or if the directions of any two processed solid line segments are different, then the cost value of any two processed solid line segments is positive infinity. Otherwise, the cost value is not positive infinity.

[0109] All cost values ​​are formed into a cost matrix. The cost matrix is matrix, The total number of solid line segments processed.

[0110] Optionally, a lane line instance is obtained according to the fitting curve and the cost matrix, including: Adjust the cost matrix according to the fitting curve, starting point coordinates and end point coordinates of each processed solid line segment.

[0111] The processed solid line segments are numbered from small to large according to the ordinate values ​​of the starting coordinates.

[0112] The processed solid line segments are merged according to the numbers and the adjusted cost matrix to obtain lane line instances.

[0113] Optionally, the cost matrix is ​​adjusted according to the fitting curve, the starting point coordinates and the end point coordinates of each processed solid line segment, including: The non-positive infinite cost values ​​in the cost matrix are determined as the cost values ​​to be adjusted.

[0114] According to the fitting curve, the starting point coordinates and the end point coordinates, the fitting error between the two processed solid line segments involved in each cost value to be adjusted is determined. and distance error .

[0115] according to and fitting error threshold The relationship between Distance error threshold The relationship between them is used to adjust the cost value to be adjusted.

[0116] Optionally, the fitting error between the two processed solid line segments involved in each cost value to be adjusted is determined according to the fitting curve, the starting point coordinates and the end point coordinates. and distance error ,include: According to the starting point coordinates of the upper line segment, a first error between the starting point of the upper line segment and the fitting curve of the lower line segment is calculated, wherein the upper line segment is one of the two lane line solid line segments involved, the lower line segment is the other of the two lane line solid line segments involved, and the upper line segment is located above the lower segment.

[0117] According to the coordinates of the end point of the lower line segment, a second error between the end point of the lower line segment and the fitting curve of the upper line segment is calculated.

[0118] Sure =first error + second error.

[0119] According to the coordinates of the starting point of the upper line segment and the coordinates of the end point of the lower line segment, the distance between the starting point of the upper line segment and the end point of the lower line segment is determined and determined as .

[0120] Optionally, according to and fitting error threshold The relationship between Distance error threshold The relationship between , the cost value to be adjusted is adjusted, including: for ,or, The cost value to be adjusted is adjusted to positive infinity.

[0121] for ,and, The cost value to be adjusted is adjusted to ,in, is the weight coefficient of the fitting error.

[0122] Optionally, the processed solid line segments are merged according to the numbers and the adjusted cost matrix to obtain lane line instances, including: All treated solid line segments are marked as untreated.

[0123] Select a processed solid line segment marked as unprocessed in order from the smallest to the largest number. If, based on the adjusted cost matrix, it is determined that there is a non-infinite cost value between the selected processed solid line segment and other processed solid line segments marked as unprocessed, then merge the selected processed solid line segment with the other processed solid line segments marked as unprocessed with the smallest cost value to obtain a merged processed solid line segment. Mark the selected processed solid line segment and other processed solid line segments marked as unprocessed with the smallest cost value as processed.

[0124] The processed solid line segments are updated to all processed solid line segments marked as unprocessed and merged processed solid line segments, and the steps of determining the fitting curve, starting point coordinates and end point coordinates of each processed solid line segment and subsequent steps are repeated until there are no new merged processed solid line segments.

[0125] The electronic device provided by this embodiment has a computer program executed by a processor to obtain a solid lane line segment, form a cost matrix according to its starting point coordinates and end point coordinates, and obtain a lane line instance according to the fitting curve and the cost matrix. An accurate lane line instance can be obtained without relying on other algorithms.

[0126] Based on the same inventive concept of the lane line instance detection method, this embodiment provides a computer-readable storage medium, and a computer program is stored thereon. The computer program is executed by a processor to implement the lane line instance detection method.

[0127] Specifically, Generate lane mask through segmentation network.

[0128] Traverse the lane line mask from bottom to top and obtain all the solid lane line segments.

[0129] Determine the direction of the solid line segment of each lane line, where the direction is horizontal or vertical.

[0130] All lane line solid line segments are determined as processed solid line segments.

[0131] Determine the fitting curve, starting point coordinates and end point coordinates of each processed solid line segment.

[0132] The cost matrix is ​​determined according to the direction, starting point coordinates and end point coordinates of each processed solid line segment.

[0133] Get the lane line instance based on the fitting curve and cost matrix.

[0134] Optionally, a cost matrix is ​​determined according to the direction, starting point coordinates and end point coordinates of each processed solid line segment, including: According to the starting point coordinates and the end point coordinates, the direction coordinate set of each processed solid line segment is determined. If the direction of the processed solid line segment is horizontal, the direction coordinate set is the set consisting of its horizontal coordinates. If the direction of the processed solid line segment is vertical, the direction coordinate set is the set consisting of its vertical coordinates.

[0135] Determine the cost value of any two processed solid line segments respectively. If the directions of any two processed solid line segments are the same and the intersection of the direction coordinate sets is not empty, or if the directions of any two processed solid line segments are different, then the cost value of any two processed solid line segments is positive infinity. Otherwise, the cost value is not positive infinity.

[0136] All cost values ​​are formed into a cost matrix. The cost matrix is matrix, The total number of solid line segments processed.

[0137] Optionally, a lane line instance is obtained according to the fitting curve and the cost matrix, including: Adjust the cost matrix according to the fitting curve, starting point coordinates and end point coordinates of each processed solid line segment.

[0138] The processed solid line segments are numbered from small to large according to the ordinate values ​​of the starting coordinates.

[0139] The processed solid line segments are merged according to the numbers and the adjusted cost matrix to obtain lane line instances.

[0140] Optionally, the cost matrix is ​​adjusted according to the fitting curve, the starting point coordinates and the end point coordinates of each processed solid line segment, including: The non-positive infinite cost values ​​in the cost matrix are determined as the cost values ​​to be adjusted.

[0141] According to the fitting curve, the starting point coordinates and the end point coordinates, the fitting error between the two processed solid line segments involved in each cost value to be adjusted is determined. and distance error .

[0142] according to and fitting error threshold The relationship between Distance error threshold The relationship between them is used to adjust the cost value to be adjusted.

[0143] Optionally, the fitting error between the two processed solid line segments involved in each cost value to be adjusted is determined according to the fitting curve, the starting point coordinates and the end point coordinates. and distance error ,include: According to the starting point coordinates of the upper line segment, a first error between the starting point of the upper line segment and the fitting curve of the lower line segment is calculated, wherein the upper line segment is one of the two lane line solid line segments involved, the lower line segment is the other of the two lane line solid line segments involved, and the upper line segment is located above the lower segment.

[0144] According to the coordinates of the end point of the lower line segment, a second error between the end point of the lower line segment and the fitting curve of the upper line segment is calculated.

[0145] Sure =first error + second error.

[0146] According to the coordinates of the starting point of the upper line segment and the coordinates of the end point of the lower line segment, the distance between the starting point of the upper line segment and the end point of the lower line segment is determined and determined as .

[0147] Optionally, according to and fitting error threshold The relationship between Distance error threshold The relationship between , the cost value to be adjusted is adjusted, including: for ,or, The cost value to be adjusted is adjusted to positive infinity.

[0148] for ,and, The cost value to be adjusted is adjusted to ,in, is the weight coefficient of the fitting error.

[0149] Optionally, the processed solid line segments are merged according to the numbers and the adjusted cost matrix to obtain lane line instances, including: All treated solid line segments are marked as untreated.

[0150] Select a processed solid line segment marked as unprocessed in order from the smallest to the largest number. If, based on the adjusted cost matrix, it is determined that there is a non-infinite cost value between the selected processed solid line segment and other processed solid line segments marked as unprocessed, then merge the selected processed solid line segment with the other processed solid line segments marked as unprocessed with the smallest cost value to obtain a merged processed solid line segment. Mark the selected processed solid line segment and other processed solid line segments marked as unprocessed with the smallest cost value as processed.

[0151] The processed solid line segments are updated to all processed solid line segments marked as unprocessed and merged processed solid line segments, and the steps of determining the fitting curve, starting point coordinates and end point coordinates of each processed solid line segment and subsequent steps are repeated until there are no new merged processed solid line segments.

[0152] The computer-readable storage medium provided in this embodiment has a computer program executed by a processor to obtain a solid lane line segment, form a cost matrix according to its starting point coordinates and end point coordinates, obtain a lane line instance according to the fitting curve and the cost matrix, and obtain an accurate lane line instance without relying on other algorithms.

[0153] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes. The schemes in the embodiments of the present application may be implemented in various computer languages, for example, object-oriented programming language Java and literal scripting language JavaScript, etc.

[0154] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0155] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0156] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0157] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of this application, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0158] Although the preferred embodiments of the present application have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0159] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A lane line instance detection method, characterized in that: The method comprises: Generate lane mask through segmentation network; Traverse the lane line mask from bottom to top to obtain all lane line solid line segments; Determine the direction of the solid line segment of each lane line; wherein the direction is horizontal or vertical; All lane line solid segments are determined as processed solid line segments; Determine the fitting curve, starting point coordinates and end point coordinates of each processed solid line segment; Determine the cost matrix according to the direction, starting point coordinates and end point coordinates of each processed solid line segment; A lane line instance is obtained according to the fitting curve and the cost matrix.

2. The method according to claim 1, characterized in that Determining the cost matrix according to the direction, starting point coordinates and end point coordinates of each processed solid line segment includes: According to the starting point coordinates and the end point coordinates, the direction coordinate set of each processed solid line segment is determined; if the direction of the processed solid line segment is horizontal, the direction coordinate set is a set consisting of its horizontal coordinates; if the direction of the processed solid line segment is vertical, the direction coordinate set is a set consisting of its vertical coordinates; Determine the cost values ​​of any two processed solid line segments respectively; wherein, if the directions of the any two processed solid line segments are the same and the intersection of the direction coordinate sets is not empty, or if the directions of the any two processed solid line segments are different, then the cost values ​​of the any two processed solid line segments are positive infinity; otherwise, the cost value is non-positive infinity; All cost values ​​are formed into a cost matrix; where the cost matrix is matrix, The total number of solid line segments processed.

3. The method according to claim 1, characterized in that The obtaining a lane line instance according to the fitting curve and the cost matrix includes: Adjust the cost matrix according to the fitting curve, the starting point coordinates and the end point coordinates of each processed solid line segment; The processed solid line segments are numbered from small to large according to the ordinate values ​​of the starting coordinates; The processed solid line segments are merged according to the numbers and the adjusted cost matrix to obtain a lane line instance.

4. The method according to claim 3, characterized in that: The step of adjusting the cost matrix according to the fitting curve, the starting point coordinates and the end point coordinates of each processed solid line segment includes: Determine the non-positive infinite cost value in the cost matrix as the cost value to be adjusted; According to the fitting curve, the starting point coordinates and the end point coordinates, the fitting error between the two processed solid line segments involved in each cost value to be adjusted is determined. and distance error ; according to and fitting error threshold The relationship between Distance error threshold The relationship between them is used to adjust the cost value to be adjusted.

5. The method according to claim 4, characterized in that The fitting error between the two processed solid line segments involved in each cost value to be adjusted is determined according to the fitting curve, the starting point coordinates and the end point coordinates. and distance error ,include: Calculate the first error between the starting point of the upper line segment and the fitting curve of the lower line segment according to the starting point coordinates of the upper line segment; wherein the upper line segment is one of the two lane line solid line segments involved, the lower line segment is the other of the two lane line solid line segments involved, and the upper line segment is located above the lower segment; Calculate the second error between the end point of the lower line segment and the fitting curve of the upper line segment according to the coordinates of the end point of the lower line segment; Sure = first error + second error; According to the coordinates of the starting point of the upper line segment and the coordinates of the end point of the lower line segment, the distance between the starting point of the upper line segment and the end point of the lower line segment is determined and determined as .

6. The method according to claim 4, characterized in that The basis and fitting error threshold The relationship between Distance error threshold The relationship between , the cost value to be adjusted is adjusted, including: for ,or, The cost value to be adjusted is adjusted to positive infinity; for ,and, The cost value to be adjusted is adjusted to ,in, is the weight coefficient of the fitting error.

7. The method according to claim 3, characterized in that The processing solid line segments are merged according to the number and the adjusted cost matrix to obtain a lane line instance, including: All treated solid line segments are marked as untreated; Select one processed solid line segment marked as unprocessed in order from the smallest to the largest number; if it is determined based on the adjusted cost matrix that there is a non-infinite cost value between the selected processed solid line segment and other processed solid line segments marked as unprocessed, then merge the selected processed solid line segment with the other processed solid line segments marked as unprocessed with the smallest cost value to obtain a merged processed solid line segment; mark the selected processed solid line segment and other processed solid line segments marked as unprocessed with the smallest cost value as processed; The processed solid line segments are updated to all processed solid line segments marked as unprocessed and merged processed solid line segments, and the steps of determining the fitting curve, starting point coordinates and end point coordinates of each processed solid line segment and subsequent steps are repeated until there are no new merged processed solid line segments.

8. A lane line instance detection device, characterized in that: The device comprises: The generation module is used to generate lane mask through the segmentation network; An acquisition module, used for traversing the lane line mask generated by the generation module from bottom to top to obtain all lane line solid line segments; A first determination module is used to determine the direction of the solid line segments of each lane line obtained by the acquisition module; wherein the direction is horizontal or vertical; A second determination module, configured to determine all lane line solid line segments acquired by the acquisition module as processed solid line segments; A third determination module, used to determine the fitting curve, the starting point coordinates and the end point coordinates of each processed solid line segment determined by the second determination module; a fourth determining module, configured to determine a cost matrix according to the directions of the processed solid line segments determined by the first determining module and the starting point coordinates and the end point coordinates determined by the third determining module; The fifth determination module is used to obtain a lane line instance according to the fitting curve determined by the third determination module and the cost matrix determined by the fourth determination module.

9. An electronic device, characterized in that: include: Memory; processor; as well as Computer programs; The computer program is stored in the memory and is configured to be executed by the processor to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: A computer program is stored thereon; the computer program is executed by a processor to implement the method according to any one of claims 1 to 7.

Images