A multi-sensor bev stitched view lane line detection method

Abstract

The application discloses a multi-sensor bev spliced view lane line detection method, including a single lane line detection method and a double lane line detection method, through lane line coding and decoding mode, various instances of lane lines with different inclination angles can be output, for double lane line detection, not only two instances of single lines can be output, but also a virtual center line can be output, and the topological relationship of the two lines can be instantiated and bound, rich lane line information is provided for downstream positioning, in addition, the detection of a stop line and the output of a lane line in one network architecture reduce the computing power, and a new idea is provided for outputting lane lines with various inclination angles.

Description

Technical Field

[0001] This invention belongs to the field of autonomous driving technology, specifically relating to a method for lane line detection using a multi-sensor BEV stitched view. Background Technology

[0002] Lane detection is a key technology in autonomous driving systems, crucial for achieving safe and reliable autonomous driving. Accurately detecting and tracking lane lines helps autonomous driving systems better understand the road environment, better pinpoint their location, and make correct decisions and driving plans, thereby improving driving safety and reliability.

[0003] Existing lane detection methods can be categorized into those based on traditional feature extraction algorithms and those based on deep learning. Traditional feature extraction methods commonly include edge detection algorithms, Hough transform algorithms, color thresholding algorithms, and model fitting algorithms. However, traditional lane detection algorithms have limitations in practical applications, such as their limited ability to handle complex road scenes, lighting variations, and occlusion. Nevertheless, they provide fundamental methods and ideas for lane detection and have played a significant role in past research and practice. With technological advancements, modern lane detection algorithms often employ deep learning and computer vision methods, enabling them to better handle complex scenes and changing conditions. The process involves first collecting image datasets containing lane lines and labeling these images to indicate the lane line locations. Then, an appropriate deep learning model architecture is selected to learn the features of the labeled lane line data, and the model is repeatedly trained to automatically identify lane lines.

[0004] There are various existing methods for representing lane lines, such as semantic segmentation-based methods, row classification-based methods, direct regression polynomial coefficient methods, and anchor-base methods. Each method has its own advantages and disadvantages: Semantic segmentation-based methods can accurately classify lane lines for each pixel, but require complex and time-consuming post-processing for the entire instantiated lane line; Row classification-based methods divide the image into a grid, generally defining a direction such as from bottom to top to learn lane lines that may belong to the same instance in each grid, which can instantiate each lane line. This is more effective for vertical lane lines from bottom to top, but often cannot output horizontal lane lines from left to right; Direct regression polynomial coefficient methods allow the model to directly learn the polynomial expression coefficients of the lane lines, but real-world scenarios are complex, and polynomials cannot fully represent them, resulting in low detection accuracy; Anchor-base methods model the instantiated lane lines globally by pre-defining anchor points, but this type of method requires pre-defined anchor points and lacks flexibility.

[0005] Lane lines, as an important element of ground markings, are an indispensable reference element in assisting vehicle positioning and route planning. A comprehensive comparison of existing instantiated lane line detection algorithms reveals that most are designed with the detection of lane lines parallel to the vehicle's direction of travel as the starting point. For lane lines perpendicular to the vehicle's direction of travel, such as stop lines, they are either output using object detection or ignored. For the former, designing a separate detection head increases the computational load, while for the latter, ignoring stop lines and other detections is not conducive to the longitudinal positioning of the vehicle. Summary of the Invention

[0006] To address the aforementioned problems, the main objective of this invention is to design a multi-sensor BEV stitched view lane line detection method, including a single-lane line detection method and a dual-lane line detection method, to solve the problems of low lane line detection accuracy, low flexibility, and high computational load in complex environments.

[0007] To achieve the above objectives, the present invention adopts the following technical solution:

[0008] A method for detecting single lane lines in a multi-sensor BEV stitched view, the method comprising a single lane line detection method, wherein the single lane line detection method includes single lane line model encoding and single lane line model decoding;

[0009] The single-lane line model encoding includes the following steps:

[0010] S11: Preprocess the multi-sensor surround view stitched bird's-eye view and establish a single-lane line mask;

[0011] S12: Traverse each single lane line mask, encode the integer points on the single lane line mask, and obtain each integer pixel point on the single lane line mask.

[0012] S13: Draw a perpendicular line from the integer pixel points on the single lane line mask to the single lane line, and calculate the coordinates of the foot of the perpendicular point;

[0013] S14: Based on the encoding principle, obtain the predecessor and successor points corresponding to the integer pixels, calculate the offset from the integer pixels to the predecessor and successor points, and obtain the regression branch of the entire feature map.

[0014] Decoding a single-lane line model includes the following steps:

[0015] S21: Traverse the feature map learned by the single lane line model, solve for the points on the single lane line mask that belong to the lane line, obtain the predecessor and successor points of the current point, as well as the predecessor point of the predecessor point and the successor point of the successor point, and recursively until the predecessor point and the successor point do not belong to the single lane line mask.

[0016] S22: End traversal, connecting all predecessor and successor points to form a single-lane line.

[0017] As a further description of the present invention, in step S11, the center lines of all single-lane lines in the figure are marked, and the points on the center lines are marked in an orderly manner. The expression of a marked single-lane line is: y i ={h i w i};

[0018] Among them, y i Representing pixel coordinates, y = {y1, ..., y2} n}={{h1,w1},...,{h n w n}};

[0019] The image is downsampled, and a mask with a fixed line width is created on the downsampled image to obtain the single-lane line mask.

[0020] As a further description of the present invention, in step S12, the encoding process first traverses all integer points on the single lane line mask to obtain each integer pixel point on the single lane line mask.

[0021] As a further description of the present invention, in step S13, a perpendicular line is drawn from any integer pixel P on the single-lane mask to the center line to obtain the foot point O of the perpendicular. Any points A and B marked on the center line containing the foot point O are selected. Then, the coordinates of the foot point O are determined by... and The perpendicular relationship is obtained; (x) B -x A )*(x O -x P )+(y B -y A )*(y O -y P ) = 0;

[0022] Where, the coordinates of point A (x A y A ), coordinates of point B (x B y B ), coordinates of point O (x O y O ), coordinates of point P (x P y P ).

[0023] As a further description of the present invention, in step S14, the points where the integer pixel point P intersects with its center line within the step radius are the predecessor point prev and the successor point next, respectively. Then the coordinates (x, y, y) of the predecessor point prev are...prev y prev The coordinates (x) of the successor point and the next point. next y next The expression is:

[0024] prev (x，y) =O (x，y) +α(O (x，y) -A (x，y) (1);

[0025] next (x，y) =O (x，y) +β(B (x，y) -O (x，y) (2);

[0026]

[0027] Where O is the foot of the perpendicular line drawn from the integer pixel point P to the center line, and A (x，y) O (x，y) B (x，y) The coordinates of points A, O, and B are respectively; step is the step radius of the predecessor and successor, α and β are coefficients, and l is the length of the corresponding line segment;

[0028] Calculate the offsets x-offset and y-offset of any integer pixel P relative to its predecessor (prev) and successor (next). The expression for x-offset is: x prev -x P The expression for y-offset is: y next -y P ;

[0029] The preceding regression branch reg_prev and the succeeding regression branch reg_next of the entire feature map are obtained.

[0030] As a further description of the present invention, in step S21, the solution for all predecessor points and successor points includes the following steps:

[0031] S211: Traverse the feature map and determine whether the current point is a point on the single lane line mask at the head of the single lane line mask;

[0032] If the probability of a single lane line is less than the threshold, skip it; if the probability of a single lane line is greater than the threshold, enter the predecessor regression branch reg_prev, obtain the offset of the current point from its corresponding predecessor point prev, and calculate the accurate predecessor point prev according to formula (1).

[0033] S212: Round the precise preceding point prev and determine whether it belongs to a point on the single lane line mask;

[0034] If so, continue to solve for the predecessor point prev corresponding to the above predecessor point prev on the predecessor regression branch reg_prev, until the predecessor point prev does not belong to the point on the single lane line mask, then find all the predecessor points prev.

[0035] S213: Traverse the feature map and determine whether the current point is a point on the single lane line mask at the head of the single lane line mask;

[0036] If the probability of a single lane line is less than the threshold, skip it; if the probability of a single lane line is greater than the threshold, enter the subsequent regression branch reg_next, obtain the offset of the current point from its corresponding successor point next, and calculate the accurate successor point next according to formula (2).

[0037] S212: Round the exact successor point next and determine whether it belongs to a point on the single lane line mask.

[0038] If so, continue solving for the successor point next corresponding to the above successor point next on the successor regression branch reg_next, until the successor point next does not belong to the point on the single lane line mask, then all successor points next are found.

[0039] As a further description of the present invention, the method also includes a method for detecting two-lane lines, wherein the method for detecting two-lane lines includes two-lane line model encoding and two-lane line model decoding.

[0040] The encoding of the two-lane line model includes the following steps:

[0041] S31: Divide any two-lane road into left and right lines according to their positional relationship in the image, calculate the virtual center line of the two-lane road using the single-lane line detection method described above, and establish a virtual center line mask;

[0042] S32: Traverse the virtual center line mask, encode the integer points on the virtual center line mask, and obtain each integer pixel point on the virtual center line mask;

[0043] S33: Using the above single-lane line detection method, calculate the coordinates of the perpendicular point on the virtual center line mask corresponding to any integer pixel point;

[0044] S34: Using the above single-lane line detection method, obtain the predecessor point and successor point corresponding to the integer pixel point respectively, and calculate the offset from the integer pixel point to the predecessor point and successor point.

[0045] S35: Draw a perpendicular line from the integer pixel point to the left line to obtain the foot of the perpendicular from the current integer pixel point to the left line, and calculate the x-offset and y-offset of the perpendicular from the current integer pixel point to the foot of the left line.

[0046] S36: Draw a perpendicular line from the integer pixel point to the right line to obtain the foot of the perpendicular from the current integer pixel point to the right line, and calculate the x-offset and y-offset of the perpendicular from the current integer pixel point to the foot of the right line.

[0047] S37: After traversing the virtual centerline mask, obtain the left regression branch offset_left and the right regression branch offset_right of the entire feature map;

[0048] Decoding a two-lane lane line model includes the following steps:

[0049] S41: Based on the single-lane line detection method described above, complete the decoding of the virtual center line of the two-lane line;

[0050] S42: Traverse each point on the virtual center line, and according to the single-lane line detection method described above, obtain the offset of the left line point and the offset of the right line point on the left line regression branch offset_left and the right line regression branch offset_right respectively;

[0051] S43: Using the offset from step S42, calculate the coordinates of the left and right line points to obtain the entire left and right lines.

[0052] Compared with the prior art, the technical advantages of the present invention are as follows:

[0053] This invention provides a multi-sensor BEV stitched view lane line detection method, including a single-lane line detection method and a dual-lane line detection method. Through lane line encoding and decoding, it can output instantiated lane lines with various tilt angles. For dual-lane line detection, it can output not only two instantiated single lines, but also a virtual center line, and can instantiate and bind the topological relationship of the two lines, providing rich lane line information for downstream positioning. In addition, the detection of stop lines and lane lines are output in the same network architecture, reducing computing power and providing a new approach to outputting lane line detection with various tilt angles. Attached Figure Description

[0054] Figure 1 This is a schematic diagram of the overall reasoning process of the method of the present invention;

[0055] Figure 2 BEV images stitched together from multiple camera views;

[0056] Figure 3 A schematic diagram illustrating the calculation of the predecessor and successor points at any point on a single-lane line mask according to the present invention;

[0057] Figure 4 This is a schematic diagram of the single-lane line model encoding process of the present invention;

[0058] Figure 5 This is a schematic diagram of the single-lane line model decoding process of the present invention;

[0059] Figure 6 This is a schematic diagram illustrating the coding principle of the dual-lane line model of the present invention;

[0060] Figure 7 This is a schematic diagram of the coding process for the two-lane line model of the present invention;

[0061] Figure 8 This is a schematic diagram of the network model structure of the present invention. Detailed Implementation

[0062] The present invention will now be described in detail with reference to the accompanying drawings:

[0063] This invention designs a lane line detection method using multi-sensor BEV stitched view, such as... Figure 1 As shown, it can adapt to detect lane lines at various angles and output all lateral lines such as stop lines. All outputs are instantiated representations, requiring no complex post-processing, providing rich ground element information for vehicle positioning systems. Specifically, it includes lane line encoding and decoding methods. During encoding, it calculates the predecessor and successor points of any point on each lane line mask within a fixed radius. During decoding, when obtaining any point on a lane line, it calculates the predecessor and successor points of the looping point until a complete lane line is connected. Figure 1 In the model, pred_reg_prev is the single-lane line precursor regression branch learned by the model, and pred_left_mask and pred_right_mask are the left and right line masks of the two-lane line learned by the model, respectively.

[0064] In one embodiment of the present invention, a method for detecting lane lines in a multi-sensor BEV stitched view is disclosed, with reference to... Figure 2-5 As shown, the method includes a single-lane line detection method, which includes single-lane line model encoding and single-lane line model decoding;

[0065] The single-lane line model encoding includes the following steps:

[0066] S11: Preprocess the multi-sensor surround view stitched bird's-eye view (surround view stitched BEV map) and establish a single lane line mask;

[0067] S12: Traverse each single lane line mask, encode the integer points on the single lane line mask, and obtain each integer pixel point on the single lane line mask.

[0068] S13: Draw a perpendicular line from the integer pixel points on the single lane line mask to the single lane line, and calculate the coordinates of the foot of the perpendicular point;

[0069] S14: Based on the encoding principle, obtain the predecessor and successor points corresponding to the integer pixels, calculate the offset from the integer pixels to the predecessor and successor points, and obtain the regression branch of the entire feature map.

[0070] Decoding a single-lane line model includes the following steps:

[0071] S21: Traverse the feature map learned by the single lane line model, solve for the points on the single lane line mask that belong to the lane line, obtain the predecessor and successor points of the current point, as well as the predecessor point of the predecessor point and the successor point of the successor point, and recursively until the predecessor point and the successor point do not belong to the single lane line mask.

[0072] S22: End traversal, all predecessor and successor points are connected to form a single lane line.

[0073] Specifically, this embodiment provides a detailed explanation of the single-lane line model encoding and decoding processes described above:

[0074] 1. Single-lane line model coding process

[0075] like Figure 2 As shown, in step S11, the stitched BEV image is viewed from all sides, and the center lines of all single lane lines in the image are marked, and the points on the center lines are marked in an orderly manner. The expression of a marked single lane line is: y i ={h i w i};

[0076] Among them, y i Representing pixel coordinates, y = {y1, ..., y2} n}={{h1,w1},...,{h n w n}};

[0077] The image is downsampled, and a mask with a fixed line width for each single-lane line is created on the downsampled image to obtain the single-lane line mask. In this embodiment, the single-lane line mask segmentation header will obtain a score indicating whether it is a lane line, that is, the probability of whether an integer point is a point on a single-lane line is learned through the single-lane line model. In addition, it should be noted that the categories of single-lane lines include solid lines, dashed lines, etc., so each point on the single-lane line mask will learn the probability of belonging to a certain category, thereby achieving the effect of distinguishing categories.

[0078] In this embodiment, the image is downsampled four times. Since the lane line width is uniform under BEV, a mask with a fixed line width is created on the downsampled image for each lane line.

[0079] In step S12, the encoding process is to first traverse all integer points on the single lane line mask to obtain each integer pixel point on the single lane line mask;

[0080] Secondly, such as Figure 3 As shown, find the predecessor and successor points of the integer pixel point P that intersect the center line within a fixed step radius. Specifically, in step S13, draw a perpendicular line from any integer pixel point P on the single-lane mask to the center line to obtain the foot point O of the perpendicular. Select any points A and B marked on the center line containing the foot point O; the coordinates of point A (x, y, y) are known. A y A ), coordinates of point B (x B y B ), coordinates of point P (x P y P If the perpendicular point O is perpendicular to the x-axis, then the coordinates of the perpendicular point O are (x-y). O y O )Depend on and The perpendicular relationship is obtained; (x) B -x A )*(x O -x P )+(y B -y A )*(y O -y P ) = 0.

[0081] In step S14, the points where the integer pixel P intersects with its centerline within a fixed step radius are the predecessor point prev and the successor point next, respectively. Then, the coordinates (x, y, y) of the predecessor point prev are... prev y prev The coordinates (x) of the successor point and the next point. next y next The expression is:

[0082] prev(x，y) =O (x，y) +α(O (x，y) -A (x，y) (1);

[0083] next (x，y) =O (x，y) +β(B (x，y) -O (x，y) (2);

[0084]

[0085] Where O is the foot of the perpendicular line drawn from the integer pixel point P to the center line, and A (x，y) O (x，y) B (x，y) Let A, O, and B be the coordinates of points A, O, and B respectively; step is the step radius of the predecessor and successor, α and β are coefficients, and l is the length of the corresponding line segment. step l is the length of step. OP l OA l OB Let be the lengths of line segments 0P, OA, and OB, respectively; calculate the x-offset and y-offset of any integer pixel P and its corresponding predecessor point prev and successor point next, where the x-offset expression is: x prev -x P The expression for y-offset is: y next -y P ; This yields the predecessor regression branch reg_prev and the successor regression branch reg_next for the entire feature map.

[0086] This embodiment discloses the principle of single-lane line model encoding through the above content, and the specific flowchart is as follows. Figure 4 As shown.

[0087] 2. Single-lane line model decoding process

[0088] like Figure 5 As shown, the above encoding structure is decoded, and a greedy recursive algorithm is used to predict the position of the current pixel point, recursively deriving its predecessor and successor points. Specifically, in step S21, the solution for all predecessor and successor points includes the following steps:

[0089] S211: Traverse the feature map and determine whether the current point is a point on the single lane line mask at the head of the single lane line mask;

[0090] If the probability of a single lane line is less than the threshold, skip it; if the probability of a single lane line is greater than the threshold, enter the predecessor regression branch reg_prev, obtain the offset of the current point from its corresponding predecessor point prev, and calculate the accurate predecessor point prev according to formula (1).

[0091] Specifically, the predecessor regression branch reg_prev will directly obtain the offset offse regression value of the corresponding predecessor point prev. Then, the offset offse plus the current point is the accurate predecessor point prev.

[0092] S212: Round the precise preceding point prev and determine whether it belongs to a point on the single lane line mask;

[0093] If so, continue to solve for the predecessor point prev corresponding to the above predecessor point prev on the predecessor regression branch reg_prev, until the predecessor point prev does not belong to the point on the single lane line mask, then find all the predecessor points prev.

[0094] S213: Traverse the feature map and determine whether the current point is a point on the single lane line mask at the head of the single lane line mask;

[0095] If the probability of a single lane line is less than the threshold, skip it; if the probability of a single lane line is greater than the threshold, enter the subsequent regression branch reg_next, obtain the offset of the current point and its corresponding next point, and calculate the accurate next point according to formula (2).

[0096] Specifically, the subsequent regression branch reg_next will directly obtain the offset offse regression value of the corresponding successor point next. Then, the offset offse plus the current point is the accurate successor point next.

[0097] S212: Round the exact successor point next and determine whether it belongs to a point on the single lane line mask.

[0098] If so, continue solving for the successor point next corresponding to the above successor point next on the successor regression branch reg_next, until the successor point next does not belong to the point on the single lane line mask, then all successor points next are found.

[0099] That is, traverse the feature map learned by the model, calculate whether the current point is a point on the single lane line mask. If the probability of the lane line is greater than the threshold, enter the predecessor regression branch reg_prev to get the offset of the current point from its predecessor point prev. According to formula (1), calculate the accurate predecessor point prev. Round the predecessor point prev and judge whether the probability belongs to a point of the lane line mask on the mask branch. If it does, continue to calculate the predecessor point prev corresponding to the above predecessor point prev on the predecessor regression branch reg_prev. Repeat this recursion until the predecessor point prev does not belong to the single lane line mask. Then all predecessor points have been found. Then use the same method to find the successor point next on the successor regression branch reg_next. Repeat the search until the end of the traversal. In this way, all points on a single lane line are searched.

[0100] In another embodiment of the present invention, a method for detecting lane lines in a multi-sensor BEV stitched view is disclosed, with reference to... Figure 6-7 As shown, the method also includes a two-lane line detection method, which includes two-lane line model encoding and two-lane line model decoding;

[0101] The encoding of the two-lane line model includes the following steps:

[0102] S31: Divide any two-lane road into left and right lines according to their positional relationship in the image, calculate the virtual center line of the two-lane road using the single-lane line detection method described above, and establish a virtual center line mask;

[0103] The distinction between the left and right lines is based on a 45-degree dividing line. When the angle is greater than 45 degrees, for the same y-coordinate compared to the x-coordinate, the line with a smaller x-coordinate is the left line, and the line with a larger x-coordinate is the right line. When the angle is less than 45 degrees, for the same x-coordinate compared to the y-coordinate, the line with a smaller y-coordinate is the left line, and the line with a larger y-coordinate is the right line.

[0104] S32: Traverse the virtual center line mask, encode the integer points on the virtual center line mask, and obtain each integer pixel point on the virtual center line mask;

[0105] S33: Using the above single-lane line detection method, calculate the coordinates of the perpendicular point on the virtual center line mask corresponding to any integer pixel point;

[0106] S34: Using the above single-lane line detection method, obtain the predecessor point and successor point corresponding to the integer pixel point respectively, and calculate the offset from the integer pixel point to the predecessor point and successor point.

[0107] S35: Draw a perpendicular line from the integer pixel point to the left line to obtain the foot of the perpendicular from the current integer pixel point to the left line, and calculate the x-offset and y-offset of the perpendicular from the current integer pixel point to the foot of the left line.

[0108] S36: Draw a perpendicular line from the integer pixel point to the right line to obtain the foot of the perpendicular from the current integer pixel point to the right line, and calculate the x-offset and y-offset of the perpendicular from the current integer pixel point to the foot of the right line.

[0109] S37: After traversing the virtual centerline mask, obtain the left regression branch offset_left and the right regression branch offset_right of the entire feature map;

[0110] Decoding a two-lane lane line model includes the following steps:

[0111] S41: Based on the single-lane line detection method described above, complete the decoding of the virtual center line of the two-lane line;

[0112] S42: Traverse each point on the virtual center line, and according to the single-lane line detection method described above, obtain the offset of the left line point and the offset of the right line point on the left line regression branch offset_left and the right line regression branch offset_right respectively;

[0113] S43: Using the offset from step S42, calculate the coordinates of the left and right line points to obtain the entire left and right lines.

[0114] Specifically, this embodiment provides a detailed explanation of the above-described encoding and decoding processes for the two-lane line model:

[0115] 3. Coding process for the two-lane line model

[0116] like Figure 6As shown, for a two-lane roadway, a clever centerline regression strategy is designed to accurately regress each line of the two-lane roadway and distinguish their categories. Since the two lines of a two-lane roadway are very close, their masks must overlap. Therefore, using the process described in steps 1 and 2 above, the virtual centerline of the two-lane roadway is first processed as a single-lane roadway. Then, the relationship between the centerline and the two single-lane roadsway is used to obtain the two single lines of the two-lane roadway, and these two single lines are processed separately. Specifically, for any two-lane roadway, the virtual centerline is calculated, a mask is applied to the centerline, and for any point on the mask, following steps 1 and 2 above, its predecessor and successor points are defined and calculated. The offsets from the predecessor and successor points are found. At the same time, the perpendiculars from the current mask point to the left and right lines of the two-lane roadway are calculated, and the offsets from the perpendiculars to the left and right lines are obtained. After traversing this process, the left-line regression branch offset_left and the right-line regression branch offset_right are obtained.

[0117] By establishing the above regression relationship, the model can not only learn the virtual center line of the two-lane road, but also obtain the two single lines of the two-lane road, providing rich positioning information for downstream applications.

[0118] 4. Decoding process of dual-lane line model

[0119] like Figure 7 As shown, firstly, based on the decoding process of the single-lane line described in section 2 above, the decoding of the virtual center line of the two-lane line is completed. Then, each point on the virtual center line is traversed, and the offsets of the left and right lines are obtained on the left regression branch offset_left and the right regression branch offset_right, respectively. Thus, the coordinates of the left and right lines are calculated in reverse order. Specifically, refer to the above formula (1), add the offsets of the left and right lines to obtain the coordinates of the left and right lines.

[0120] The above content discloses a multi-sensor BEV stitched view lane line detection method of the present invention, such as... Figure 8 The image shown is a view of the model's mesh structure. Compared to existing technologies, this invention has the following advantages:

[0121] 1. The detection method of the present invention includes lane line encoding and decoding processes. It is ingeniously and simply designed and can output instantiated lane lines with various tilt angles, including stop lines, providing rich positioning information for downstream applications.

[0122] 2. The detection method of the present invention can output lane line category information, color information, and occlusion information with instance information;

[0123] 3. The detection method of the present invention includes the detection of single lane lines and the detection of dual lane lines. For dual lane lines, it can not only output two instantiated lane lines, but also output a virtual center line, and can instantiate and bind the topological relationship of the two lines, providing rich positioning information for downstream applications.

[0124] 4. The detection method of the present invention outputs the detection of stop lines and lane lines in the same network architecture, reducing computing power.

[0125] The above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Any other modifications or equivalent substitutions made by those skilled in the art to the technical solutions of the present invention, as long as they do not depart from the spirit and scope of the technical solutions of the present invention, should be covered within the scope of the claims of the present invention.

Claims

1. A method for detecting lane lines in a multi-sensor BEV stitched view, characterized in that: The method includes a single-lane line detection method, which includes single-lane line model encoding and single-lane line model decoding; The single-lane line model encoding includes the following steps: S11: Preprocess the multi-sensor surround view stitched bird's-eye view and establish a single-lane line mask; S12: Traverse each single lane line mask, encode the integer points on the single lane line mask, and obtain each integer pixel point on the single lane line mask. S13: Draw a perpendicular line from the integer pixel points on the single lane line mask to the single lane line, and calculate the coordinates of the foot of the perpendicular point; S14: Based on the encoding principle, obtain the predecessor and successor points corresponding to the integer pixels, calculate the offset from the integer pixels to the predecessor and successor points, and obtain the predecessor and successor regression branches of the entire feature map. Decoding a single-lane line model includes the following steps: S21: Traverse the feature map learned by the single lane line model, solve for the points on the single lane line mask that belong to the lane line, and use the predecessor regression branch and the successor regression branch to solve for the predecessor point and successor point of the current point, as well as the predecessor point of the predecessor point and the successor point of the successor point. Repeat the recursion until the predecessor point and the successor point do not belong to the single lane line mask. S22: End traversal, all predecessor and successor points are connected to form a single lane line.

2. The method for detecting lane lines in a multi-sensor BEV stitched view according to claim 1, characterized in that: In step S11, the center lines of all single-lane lines in the diagram are marked, and the points on the center lines are marked in an orderly manner. The expression of a marked single-lane line is: ; in, Represents pixel coordinates, ; The image is downsampled, and a mask with a fixed line width is created on the downsampled image to obtain the single-lane line mask.

3. The method for lane line detection in a multi-sensor BEV stitched view according to claim 1, characterized in that: In step S12, the encoding process first involves traversing all integer points on the single-lane line mask to obtain each integer pixel point on the single-lane line mask.

4. The method for detecting lane lines in a multi-sensor BEV stitched view according to claim 2, characterized in that: In step S13, a perpendicular line is drawn from any integer pixel P on the single-lane mask to the center line to obtain the foot point O. Any points A and B marked on the center line containing the foot point O are selected. The coordinates of the foot point O are then determined by... and The vertical relationship is obtained; ； Where, the coordinates of point A Coordinates of point B Coordinates of point O Coordinates of point P .

5. The method for detecting lane lines in a multi-sensor BEV stitched view according to claim 4, characterized in that: In step S14, the points where the integer pixel P intersects with its centerline within the step radius are the predecessor point prev and the successor point next, respectively. Then the coordinates of the predecessor point prev are... and the coordinates of the successor point next The expression is: （1）； （2）； ， ； Where O is the foot of the perpendicular line drawn from the integer pixel point P to the center line. , , , where A, O, and B are the coordinates of points respectively; step is the step size radius of the predecessor and successor. , They are coefficients, This represents the length of the corresponding line segment; Calculate the offsets x-offset and y-offset of any integer pixel P relative to its predecessor (prev) and successor (next). The expression for x-offset is: The expression for y-offset is: ; The preceding regression branch reg_prev and the succeeding regression branch reg_next of the entire feature map are obtained.

6. The method for detecting lane lines in a multi-sensor BEV stitched view according to claim 5, characterized in that: In step S21, the solution for all predecessor and successor points includes the following steps: S211: Traverse the feature map and determine whether the current point is a point on the single lane line mask at the head of the single lane line mask; If the probability of a single lane line is less than the threshold, skip it; if the probability of a single lane line is greater than the threshold, enter the predecessor regression branch reg_prev, obtain the offset of the current point from its corresponding predecessor point prev, and calculate the accurate predecessor point prev according to formula (1). S212: Round the precise preceding point prev and determine whether it belongs to a point on the single lane line mask; If so, continue to solve for the predecessor point prev corresponding to the above predecessor point prev on the predecessor regression branch reg_prev, until the predecessor point prev does not belong to the point on the single lane line mask, then find all the predecessor points prev. S213: Traverse the feature map and determine whether the current point is a point on the single lane line mask at the head of the single lane line mask; If the probability of a single lane line is less than the threshold, skip it; if the probability of a single lane line is greater than the threshold, enter the subsequent regression branch reg_next, obtain the offset of the current point from its corresponding successor point next, and calculate the accurate successor point next according to formula (2). S212: Round the exact successor point next and determine whether it belongs to a point on the single lane line mask. If so, continue solving for the successor point next corresponding to the above successor point next on the successor regression branch reg_next, until the successor point next does not belong to the point on the single lane line mask, then all successor points next are found.

7. The method for detecting lane lines in a multi-sensor BEV stitched view according to claim 5, characterized in that: The method also includes a method for detecting two-lane lines, wherein the method for detecting two-lane lines includes two-lane line model encoding and two-lane line model decoding; The encoding of the two-lane line model includes the following steps: S31: Divide any two-lane road into left and right lines according to their positional relationship in the image, calculate the virtual center line of the two-lane road using the single-lane line detection method described above, and establish a virtual center line mask; S32: Traverse the virtual center line mask, encode the integer points on the virtual center line mask, and obtain each integer pixel point on the virtual center line mask; S33: Using the above single-lane line detection method, calculate the coordinates of the perpendicular point on the virtual center line mask corresponding to any integer pixel point; S34: Using the above single-lane line detection method, obtain the predecessor point and successor point corresponding to the integer pixel point respectively, and calculate the offset from the integer pixel point to the predecessor point and successor point. S35: Draw a perpendicular line from the integer pixel point to the left line to obtain the foot of the perpendicular from the current integer pixel point to the left line, and calculate the x-offset and y-offset of the perpendicular from the current integer pixel point to the foot of the left line. S36: Draw a perpendicular line from the integer pixel point to the right line to obtain the foot of the perpendicular from the current integer pixel point to the right line, and calculate the x-offset and y-offset of the perpendicular from the current integer pixel point to the foot of the right line. S37: After traversing the virtual centerline mask, obtain the left regression branch offset_left and the right regression branch offset_right of the entire feature map; Decoding a two-lane lane line model includes the following steps: S41: Based on the single-lane line detection method described above, complete the decoding of the virtual center line of the two-lane line; S42: Traverse each point on the virtual center line, and according to the single-lane line detection method described above, obtain the offset of the left line point and the offset of the right line point on the left line regression branch offset_left and the right line regression branch offset_right respectively; S43: Using the offset from step S42, calculate the coordinates of the left and right line points to obtain the entire left and right lines.

Images