Multi-sensor bev splicing view lane line detection method

Abstract

The invention discloses a multi-sensor bev splicing view lane line detection method, which comprises a single lane line detection method and a double lane line detection method, can output instantiated lane lines with various inclination angles through lane line coding and decoding modes, and can output two instantiated single lines and two instantiated lane lines for double lane line detection. According to the invention, the method can achieve the detection of the stop line, also can output a virtual center line, can instantiate and bind the topological relation of two lines, provides abundant lane line information for downstream positioning, reduces the calculation power for the detection of the stop line and the output of the lane line in a network architecture, and provides a new thought for the detection of the lane lines with various inclination angles.

Description

Technical Field

[0001] The present invention belongs to the technical field of autonomous driving, and in particular relates to a lane line detection method of a multi-sensor BEV splicing view. Background Art

[0002] Lane detection is a key technology in autonomous driving systems and is essential for achieving safe and reliable autonomous driving. Accurately detecting and tracking lane lines can help autonomous driving systems better understand the road environment, better locate their own position, make correct decisions and travel plans, thereby improving driving safety and reliability.

[0003] Existing lane detection methods are divided into those based on traditional feature extraction algorithms and those based on deep learning. Common traditional feature extraction methods include edge detection algorithms, Hough transform algorithms, color threshold segmentation algorithms, model fitting algorithms, etc. Traditional lane detection algorithms have certain limitations in practical applications, such as limited processing capabilities for complex road scenes, lighting changes, and occlusions. However, they provide some basic methods and ideas for lane detection and have played an important role in past research and practice. With the development of technology, modern lane detection algorithms often use deep learning and computer vision methods to better cope with complex scenes and changing conditions: First, collect image data sets containing lane lines, and annotate these images to mark the locations of lane lines. Select an appropriate deep learning model architecture to learn the annotated lane line data features, and repeatedly train the model to automatically identify lane lines.

[0004] There are many ways to express lane lines, such as semantic segmentation-based methods, row classification-based methods, direct regression polynomial coefficient methods, anchor_base methods, etc. These methods have their own advantages and disadvantages: the semantic segmentation-based method can accurately obtain the lane line classification of each pixel, but for the entire instantiated lane line, complex and time-consuming post-processing is required; the row classification-based method divides the image into a grid, and generally stipulates a direction such as learning the lane lines that may belong to the same instance in each grid from bottom to top, and can instantiate each lane line. It is more friendly to vertical lane lines from bottom to top, but often cannot output horizontal lane lines from left to right; the direct regression polynomial coefficient method allows the model to directly learn the lane line fitting polynomial expression coefficients, but the actual scene is complex and the polynomial cannot be fully expressed, so the detection accuracy is not high; the anchor_base method globally models the instantiated lane line by presetting anchor points. This type of method requires predefined anchor points and is not flexible enough.

[0005] Lane lines are important elements of ground markings and are indispensable reference elements for assisting vehicle positioning and route planning. A comprehensive comparison of existing instantiated lane line detection algorithms shows that most of them are designed based on detecting lane lines parallel to the vehicle's travel direction. Lane lines such as stop lines that are perpendicular to the vehicle's travel direction are either output using target detection or ignored. For the former, designing a separate detection head increases the amount of calculation. For the latter, ignoring detection of stop lines and other detections is not conducive to the longitudinal positioning of the vehicle. Summary of the invention

[0006] In view of the above problems, the main purpose of the present invention is to design a multi-sensor BEV stitching view lane line detection method, including a single lane line detection method and a double lane line detection method, to solve the problems of low lane line detection accuracy, low flexibility and large computational complexity in complex environments.

[0007] In order to achieve the above purpose, the present invention adopts the following technical solutions:

[0008] A multi-sensor BEV stitching view single lane line detection method, the method comprising a single lane line detection method, the single lane line detection method comprising single lane line model encoding and single lane line model decoding;

[0009] The single lane line model encoding includes the following steps:

[0010] S11: Preprocess the bird's-eye view stitching of the multi-sensor surround view and establish a single lane line mask;

[0011] S12: traverse each single lane line mask, encode the integer points on the single lane line mask, and obtain each integer pixel point on the single lane line mask;

[0012] S13: Draw a perpendicular line from the integer pixel points on the single lane line mask to the single lane line, and calculate the coordinates of the foot point of the perpendicular line;

[0013] S14: Obtain the predecessor point and the successor point corresponding to the integer pixel point according to the coding principle, calculate the offset from the integer pixel point to the predecessor point and the successor point, and obtain the regression branch of the entire feature map;

[0014] Single lane line model decoding includes the following steps:

[0015] S21: Traverse the feature map learned by the single lane line model, solve the points on the single lane line mask that belong to the lane line, obtain the predecessor point and successor point of the current point, as well as the predecessor point of the predecessor point and the successor point of the successor point, and repeatedly recursively until the predecessor point and the successor point do not belong to the single lane line mask;

[0016] S22: End the traversal, and connect all the predecessor points and successor points into a single lane line.

[0017] As a further description of the present invention, in step S11, all the single lane lines in the figure are marked with their center lines, and the points on the center lines are marked in order. The expression of a marked single lane line is: i ={h i , w i};

[0018] Among them, y i represents pixel coordinates, y = {y1, ..., y n}={{h1,w1},...,{h n , w n}};

[0019] The image is downsampled, and a mask with a fixed line width for each single lane line is created on the downsampled image to obtain a single lane line mask.

[0020] As a further description of the present invention, in step S12, the encoding process first traverses all integer points on the single lane line mask to obtain each integer pixel point on the single lane line mask.

[0021] As a further description of the present invention, in step S13, a perpendicular line is drawn from any integer pixel point P on the single lane line mask to the center line to obtain a perpendicular foot point O, and any points A and B marked on the center line including the perpendicular foot point O are selected. The coordinates of the perpendicular foot point O are given by and The vertical relationship is obtained; (x B -x A )*(x O -x P )+(y B -y A )*(y O -y P )=0;

[0022] Among them, the coordinates of point A (x A ,y A ), coordinates of point B (x B ,y B ), coordinates of point O (x O ,y O ), coordinates of point P (x P ,y P ).

[0023] As a further description of the present invention, in step S14, the points where the integer pixel point P intersects with the center line within the step radius are the predecessor point prev and the successor point next, then the coordinates (xprev ,y prev ) and the coordinates of the successor point next (x next ,y next ), the expression is:

[0024] prev (x，y) =O (x，y) +α(O (x，y) -A (x，y) ) (1);

[0025] next (x，y) =O (x，y) +β(B (x，y) -O (x，y) ) (2);

[0026]

[0027] Among them, O is the foot point of the perpendicular line from the integer pixel point P to the center line, A (x，y) , O (x，y) , B (x，y) are the coordinates of point A, point O, and point B respectively; step is the step radius of the predecessor and successor, α and β are the coefficients respectively, and l is the length of the corresponding line segment;

[0028] Calculate the offset x-offset and y-offset of any integer pixel point P and its corresponding predecessor point prev and successor point next. The x-offset expression is: prev -x P , the expression of y-offset is: next -y P ;

[0029] Get the predecessor regression branch reg_prev and the successor regression branch reg_next of the entire feature graph.

[0030] As a further description of the present invention, in step S21, the solution of all predecessor points and successor points includes the following steps:

[0031] S211: traverse the feature map and determine whether the current point is a point on the single lane line mask.

[0032] If the probability of a single lane line is less than the threshold, skip it; if the probability of a single lane line is greater than the threshold, enter the predecessor regression branch reg_prev, obtain the offset of the current point from its corresponding predecessor point prev, and inversely calculate the precise predecessor point prev according to formula (1);

[0033] S212: rounding the precise predecessor point prev, and determining whether it belongs to a point on the single lane line mask on the single lane line mask;

[0034] If yes, continue to solve the predecessor point prev corresponding to the above predecessor point prev on the predecessor regression branch reg_prev until the predecessor point prev does not belong to the point on the single lane line mask, and then find all the predecessor points prev;

[0035] S213: traverse the feature map and determine whether the current point is a point on the single lane line mask.

[0036] If the probability of a single lane line is less than the threshold, skip it; if the probability of a single lane line is greater than the threshold, enter the subsequent regression branch reg_next, obtain the offset of the current point from its corresponding successor point next, and inversely calculate the exact successor point next according to formula (2);

[0037] S212: rounding the precise successor point next, and determining whether it belongs to a point on the single lane line mask on the single lane line mask;

[0038] If so, continue to solve the successor point next corresponding to the above successor point next on the successor regression branch reg_next until the successor point next does not belong to the point on the single lane line mask, and then find all the successor points next.

[0039] As a further description of the present invention, the method also includes a double lane line detection method, the double lane line detection method double lane line model encoding and double lane line model decoding;

[0040] The dual lane line model encoding includes the following steps:

[0041] S31: distinguishing any two-lane line into a left line and a right line according to the positional relationship on the image, calculating a virtual center line of the two-lane line by the above-mentioned single-lane line detection method, and establishing a virtual center line mask;

[0042] S32: traverse the virtual center line mask, encode the integer points on the virtual center line mask, and obtain each integer pixel point on the virtual center line mask;

[0043] S33: by using the above single lane line detection method, the coordinates of the perpendicular points on the virtual center line mask corresponding to any integer pixel points are calculated respectively;

[0044] S34: Obtain the predecessor point and the successor point corresponding to the integer pixel point through the above-mentioned single lane line detection method, and calculate the offset from the integer pixel point to the predecessor point and the successor point;

[0045] S35: Draw a perpendicular line from the integer pixel point to the left line to obtain the foot point of the perpendicular from the current integer pixel point to the left line, and calculate the x-offset and y-offset from the current integer pixel point to the foot point of the perpendicular from the left line;

[0046] S36: Draw a perpendicular line from the integer pixel point to the right line to obtain the foot point of the perpendicular from the current integer pixel point to the right line, and calculate the x-offset and y-offset from the current integer pixel point to the foot point of the perpendicular from the right line;

[0047] S37: After traversing the virtual center line mask, the left line regression branch offset_left and the right line regression branch offset_right of the entire feature map are obtained;

[0048] The two-lane line model decoding includes the following steps:

[0049] S41: According to the above-mentioned single lane line detection method, complete the decoding of the virtual center line of the double lane line;

[0050] S42: traverse each point on the virtual center line, and obtain the offset of the left line point and the offset of the right line point on the left line regression branch offset_left and the right line regression branch offset_right respectively according to the above single lane line detection method;

[0051] S43: Using the offset in step S42, the coordinates of the left line point and the right line point are inversely calculated to obtain the entire left line and right line.

[0052] Compared with the prior art, the technical effects of the present invention are:

[0053] The present invention provides a lane line detection method for a multi-sensor BEV splicing view, including a single lane detection method and a double lane line detection method. Through the encoding and decoding of lane lines, instantiated lane lines of various inclination angles can be output. For the detection of double lane lines, not only two instantiated single lines can be output, but also a virtual center line can be output, and the topological relationship of the two lines can be instantiated and bound, providing rich lane line information for downstream positioning. In addition, the detection of stop lines and lane lines are output in a network architecture to reduce computing power, providing a new idea for outputting lane line detection of various inclination angles. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 It is a schematic diagram of the overall reasoning process of the method of the present invention;

[0055] Figure 2 BEV images fused and stitched by multi-view cameras;

[0056] Figure 3 Schematic diagram of calculating the predecessor point and the successor point of any point on the single lane line mask of the present invention;

[0057] Figure 4 A schematic diagram of the encoding process of a single lane line model of the present invention;

[0058] Figure 5 A schematic diagram of the decoding process of a single lane line model of the present invention;

[0059] Figure 6 It is a schematic diagram of the encoding principle of the dual lane line model of the present invention;

[0060] Figure 7 It is a schematic diagram of the encoding process of the dual lane line model of the present invention;

[0061] Figure 8 It is a schematic diagram of the network model structure of the present invention. DETAILED DESCRIPTION

[0062] The present invention is described in detail below in conjunction with the accompanying drawings:

[0063] The present invention designs a multi-sensor BEV splicing view lane line detection method, such as Figure 1 As shown in the figure, it can adapt to detect lane lines of various angles, and can output all horizontal lines such as stop lines. The outputs are all instantiated expressions, without the need for complex post-processing, providing rich ground element information for the vehicle positioning system. Specifically, it includes lane line encoding and decoding methods. When encoding, the predecessor and successor points of any point on each lane line mask within a fixed radius are calculated. When decoding, when any point on a lane line is obtained, the predecessor and successor points of this point are calculated and looped until a complete lane line is connected. Figure 1 In the figure, pred_reg_prev is the single lane line precursor regression branch learned by the model, and pred_left_mask and pred_right_mask are the left line mask and right line mask of the double lane line learned by the model respectively.

[0064] In one embodiment of the present invention, a multi-sensor BEV stitching view lane line detection method is disclosed, referring to Figure 2-5 As shown, the method includes a single lane line detection method, and the single lane line detection method includes single lane line model encoding and single lane line model decoding;

[0065] The single lane line model encoding includes the following steps:

[0066] S11: Preprocess the surround view stitching bird's eye view image (surround view stitching BEV image) of multiple sensors and establish a single lane line mask;

[0067] S12: traverse each single lane line mask, encode the integer points on the single lane line mask, and obtain each integer pixel point on the single lane line mask;

[0068] S13: Draw a perpendicular line from the integer pixel points on the single lane line mask to the single lane line, and calculate the coordinates of the foot point of the perpendicular line;

[0069] S14: Obtain the predecessor point and the successor point corresponding to the integer pixel point according to the coding principle, calculate the offset from the integer pixel point to the predecessor point and the successor point, and obtain the regression branch of the entire feature map;

[0070] Single lane line model decoding includes the following steps:

[0071] S21: Traverse the feature map learned by the single lane line model, solve the points on the single lane line mask that belong to the lane line, obtain the predecessor point and successor point of the current point, as well as the predecessor point of the predecessor point and the successor point of the successor point, and repeatedly recursively until the predecessor point and the successor point do not belong to the single lane line mask;

[0072] S22: End the traversal, and connect all the predecessor points and successor points into a single lane line.

[0073] Specifically, this embodiment describes in detail the above-mentioned single lane line model encoding process and single lane line model decoding process:

[0074] 1. Single lane line model encoding process

[0075] like Figure 2 As shown, in step S11, the stitched BEV image is looked around, and the center lines of all the single lane lines in the image are marked, and the points on the center lines are marked in order. The expression of a marked single lane line is: i ={h i , w i};

[0076] Among them, y i represents pixel coordinates, y = {y1, ..., y n}={{h1,w1},...,{h n , w n}};

[0077] The image is downsampled, and a mask with a fixed line width for each single lane line is established on the downsampled image to obtain a single lane line mask. In this embodiment, the score of whether it is a lane line can be obtained on the single lane line mask segmentation head, that is, the probability of whether an integer point is a point on the single lane line is learned through the single lane line model. In addition, it should be noted that the categories of single lane lines include solid lines, dotted lines, etc., and each point on the single lane line mask will learn the probability of belonging to a certain category, so as to achieve the effect of distinguishing categories.

[0078] In this embodiment, the image is downsampled four times. Since the width of the lane lines in the BEV is uniform, a mask with a fixed width for each lane line is established on the downsampled image.

[0079] In step S12, the encoding process is to first traverse all integer points on the single lane line mask to obtain each integer pixel point on the single lane line mask;

[0080] Secondly, if Figure 3 As shown in FIG. 1 , find the predecessor and successor points where the integer pixel point P intersects with the center line within a fixed step radius. Specifically, in step S13, draw a perpendicular line from any integer pixel point P on the single lane line mask to the center line to obtain the foot point O, and select any points A and B marked on the center line containing the foot point O; the coordinates of point A (x A ,y A ), coordinates of point B (x B ,y B ), coordinates of point P (x P ,y P ), then the coordinates of the foot point O (x O ,y O )Depend on and The vertical relationship is obtained; (x B -x A )*(x O -x P )+(y B -y A )*(y O -y P )=0.

[0081] In step S14, the points where the integer pixel point P intersects with its center line within the fixed step radius are the predecessor point prev and the successor point next, respectively. The coordinates (x prev ,y prev ) and the coordinates of the successor point next (x next ,y next ), the expression is:

[0082] prev(x，y) =O (x，y) +α(O (x，y) -A (x，y) ) (1);

[0083] next (x，y) =O (x，y) +β(B (x，y) -O (x，y) ) (2);

[0084]

[0085] Among them, O is the foot point of the perpendicular line from the integer pixel point P to the center line, A (x，y) , O (x，y) , B (x，y) are the coordinates of point A, point O, and point B respectively; step is the step radius of the predecessor and successor, α and β are coefficients respectively, l is the length of the corresponding line segment, l step is the length of step, l OP , l OA , l OB are the lengths of line segments 0P, OA, and OB respectively; calculate the x-offset and y-offset of any integer pixel point P and its corresponding predecessor point prev and successor point next. The x-offset expression is: prev -x P , the expression of y-offset is: next -y P ; Get the predecessor regression branch reg_prev and the successor regression branch reg_next of the entire feature graph.

[0086] This embodiment discloses the principle of single lane line model encoding through the above content, and the specific flow chart is as follows: Figure 4 shown.

[0087] 2. Single lane line model decoding process

[0088] like Figure 5 As shown, the above coding structure is decoded, and a greedy recursive algorithm is used for the predicted position of the current pixel point to recursively deduce the predecessor points and successor points connected to it. Specifically, in step S21, the solution of all predecessor points and successor points includes the following steps:

[0089] S211: traverse the feature map and determine whether the current point is a point on the single lane line mask.

[0090] If the probability of a single lane line is less than the threshold, skip it; if the probability of a single lane line is greater than the threshold, enter the predecessor regression branch reg_prev, obtain the offset of the current point from its corresponding predecessor point prev, and inversely calculate the precise predecessor point prev according to formula (1);

[0091] Specifically, the predecessor regression branch reg_prev will directly obtain the offset offset regression value of the corresponding predecessor point prev, and the offset offset plus the current point is the precise predecessor point prev;

[0092] S212: rounding the precise predecessor point prev, and determining whether it belongs to a point on the single lane line mask on the single lane line mask;

[0093] If yes, continue to solve the predecessor point prev corresponding to the above predecessor point prev on the predecessor regression branch reg_prev until the predecessor point prev does not belong to the point on the single lane line mask, and then find all the predecessor points prev;

[0094] S213: traverse the feature map and determine whether the current point is a point on the single lane line mask.

[0095] If the probability of a single lane line is less than the threshold, skip it; if the probability of a single lane line is greater than the threshold, enter the subsequent regression branch reg_next, obtain the offset between the current point and its corresponding successor point next, and inversely calculate the exact successor point next according to formula (2);

[0096] Specifically, the successor regression branch reg_next will directly obtain the offset offse regression value corresponding to the successor point next, and the offset offse plus the current point is the precise successor point next;

[0097] S212: rounding the precise successor point next, and determining whether it belongs to a point on the single lane line mask on the single lane line mask;

[0098] If so, continue to solve the successor point next corresponding to the above successor point next on the successor regression branch reg_next until the successor point next does not belong to the point on the single lane line mask, and then find all the successor points next.

[0099] That is, traverse the feature map learned by the model, calculate whether the current point is a point on the single lane line mask, if the probability of the lane line is greater than the threshold, enter the predecessor regression branch reg_prev to obtain the offset of the current point from its predecessor point prev, and reversely calculate the precise predecessor point prev according to formula (1). Round the predecessor point prev and judge whether the probability belongs to a certain point of the lane line mask on the mask branch. If so, continue to calculate the predecessor point prev corresponding to the above predecessor point prev on the predecessor regression branch reg_prev, and repeat the recursion until the predecessor point prev does not belong to the single lane line mask. Then all the predecessor points are found. Then use the same method to find the successor point next on the successor regression branch reg_next, and repeat the search until the traversal ends. In this way, the points on the entire single lane line are searched.

[0100] In another embodiment of the present invention, a multi-sensor BEV stitching view lane line detection method is disclosed, referring to Figure 6-7 As shown, the method also includes a double lane line detection method, and the double lane line detection method includes double lane line model encoding and double lane line model decoding;

[0101] The dual lane line model encoding includes the following steps:

[0102] S31: distinguishing any two-lane line into a left line and a right line according to the positional relationship on the image, calculating a virtual center line of the two-lane line by the above-mentioned single-lane line detection method, and establishing a virtual center line mask;

[0103] The above distinction between the left line and the right line is based on 45 degrees. When the y coordinate is greater than 45 degrees, the smaller x coordinate is the left line, and the larger x coordinate is the right line. When the x coordinate is less than 45 degrees, the smaller y coordinate is the left line, and the larger y coordinate is the right line.

[0104] S32: traverse the virtual center line mask, encode the integer points on the virtual center line mask, and obtain each integer pixel point on the virtual center line mask;

[0105] S33: by using the above single lane line detection method, the coordinates of the perpendicular points on the virtual center line mask corresponding to any integer pixel points are calculated respectively;

[0106] S34: Obtain the predecessor point and the successor point corresponding to the integer pixel point through the above-mentioned single lane line detection method, and calculate the offset from the integer pixel point to the predecessor point and the successor point;

[0107] S35: Draw a perpendicular line from the integer pixel point to the left line to obtain the foot point of the perpendicular from the current integer pixel point to the left line, and calculate the x-offset and y-offset from the current integer pixel point to the foot point of the perpendicular from the left line;

[0108] S36: Draw a perpendicular line from the integer pixel point to the right line to obtain the foot point of the perpendicular from the current integer pixel point to the right line, and calculate the x-offset and y-offset from the current integer pixel point to the foot point of the perpendicular from the right line;

[0109] S37: After traversing the virtual center line mask, the left line regression branch offset_left and the right line regression branch offset_right of the entire feature map are obtained;

[0110] The two-lane line model decoding includes the following steps:

[0111] S41: According to the above-mentioned single lane line detection method, complete the decoding of the virtual center line of the double lane line;

[0112] S42: traverse each point on the virtual center line, and obtain the offset of the left line point and the offset of the right line point on the left line regression branch offset_left and the right line regression branch offset_right respectively according to the above single lane line detection method;

[0113] S43: Using the offset in step S42, the coordinates of the left line point and the right line point are inversely calculated to obtain the entire left line and right line.

[0114] Specifically, this embodiment describes in detail the above-mentioned two-lane line model encoding process and two-lane line model decoding process:

[0115] 3. Dual-lane line model encoding process

[0116] like Figure 6As shown in the figure, for the two-lane line, a centerline regression strategy is cleverly designed to accurately regress each line of the two-lane line and distinguish the categories. Since the two lines of the two-lane line are very close, the masks of the two lines must overlap. Therefore, using the above 1 and 2 processes, the virtual centerline of the two-lane line is first processed as a single-lane line, and then the relationship between the centerline and the two single-lane lines is used to obtain the two single lines of the two-lane line, and the two single lines are processed separately. The specific method is: for any two-lane line, the virtual centerline is calculated, the centerline is masked, and for any point on the mask, according to the above 1 and 2 processes, its predecessor and successor points are defined and calculated, and the offset with the predecessor and successor points is found. At the same time, the current mask point is calculated to the foot point of the left line and the foot point of the right line of the two-lane line, and its offset with the foot point of the left line and the foot point of the right line is obtained. After traversing like this, the left line regression branch offset_left and the right line regression branch offset_right are obtained.

[0117] By establishing the above regression relationship, the model can not only learn the virtual center line of the two-lane line, but also obtain the two single lines of the two-lane line, providing rich positioning information for the downstream.

[0118] 4. Dual-lane line model decoding process

[0119] like Figure 7 As shown, first, according to the decoding process of the single lane line in the above 2, the decoding of the virtual center line of the double lane line is completed, and then each point on the virtual center line is traversed, and the offset of the left line point and the offset of the right line point are obtained on the left line regression branch offset_left and the right line regression branch offset_right, respectively, so as to reversely calculate the coordinates of the left line point and the right line point in turn. Specifically, refer to the above formula (1), add the offset of the left line point and the offset of the right line point, and obtain the coordinates of the left line point and the right line point.

[0120] Through the above content, a multi-sensor BEV splicing view lane line detection method of the present invention is disclosed, such as Figure 8 The figure shows the grid structure view of the model. Compared with the prior art, the present invention has the following advantages:

[0121] 1. The detection method of the present invention includes the lane line encoding and decoding process, which is cleverly and concisely designed and can output instantiated lane lines of various inclination angles, including the output of stop lines, providing rich positioning information for the downstream;

[0122] 2. The detection method of the present invention can output the category information, color information, and occlusion information of the lane line with instance information;

[0123] 3. The detection method of the present invention includes the detection of single lanes and double lanes. For double lanes, it can not only output two instantiated lane lines, but also output a virtual center line, and can instantiate the topological relationship between the two lines, providing rich positioning information for the downstream;

[0124] 4. The detection method of the present invention outputs the detection of the stop line and the lane line in one network architecture, thus reducing computing power.

[0125] The above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Other modifications or equivalent substitutions made to the technical solution of the present invention by ordinary technicians in this field should be included in the scope of the claims of the present invention as long as they do not depart from the spirit and scope of the technical solution of the present invention.

Claims

1. A lane line detection method for a multi-sensor BEV splicing view, characterized by: The method includes a single lane line detection method, and the single lane line detection method includes single lane line model encoding and single lane line model decoding; The single lane line model encoding includes the following steps: S11: Preprocess the bird's-eye view stitching of the multi-sensor surround view and establish a single lane line mask; S12: traverse each single lane line mask, encode the integer points on the single lane line mask, and obtain each integer pixel point on the single lane line mask; S13: Draw a perpendicular line from the integer pixel points on the single lane line mask to the single lane line, and calculate the coordinates of the foot point of the perpendicular line; S14: Obtain the predecessor point and the successor point corresponding to the integer pixel point according to the coding principle, calculate the offset from the integer pixel point to the predecessor point and the successor point, and obtain the regression branch of the entire feature map; Single lane line model decoding includes the following steps: S21: Traverse the feature map learned by the single lane line model, solve the points on the single lane line mask that belong to the lane line, obtain the predecessor point and successor point of the current point, as well as the predecessor point of the predecessor point and the successor point of the successor point, and repeatedly recursively until the predecessor point and the successor point do not belong to the single lane line mask; S22: End the traversal, and connect all the predecessor points and successor points into a single lane line.

2. A multi-sensor BEV splicing view lane line detection method according to claim 1, characterized in that: In step S11, all the single lane lines in the figure are marked with their center lines, and the points on the center lines are marked in order. The expression of a marked single lane line is: i ={h i , w i }; Among them, y i represents pixel coordinates, y={y1,…,y n }={{h1,w1},…,{h n , w n }}; The image is downsampled, and a mask with a fixed line width for each single lane line is created on the downsampled image to obtain a single lane line mask.

3. The multi-sensor BEV splicing view lane line detection method according to claim 1, characterized in that: In step S12, the encoding process first traverses all integer points on the single lane line mask to obtain each integer pixel point on the single lane line mask.

4. The multi-sensor BEV splicing view lane line detection method according to claim 2, characterized in that: In step S13, a perpendicular line is drawn from any integer pixel point P on the single lane line mask to the center line to obtain the foot point O. Any points A and B marked on the center line including the foot point O are selected. The coordinates of the foot point O are given by and The vertical relationship is obtained; (X B -X A )*(X O -X P )+(and B -and A )*(and O -and P )=0; Among them, the coordinates of point A (x A ,y A ), coordinates of point B (x B ,y B ), coordinates of point O (x O ,y O ), coordinates of point P (x P ,y P ) 。 5. The multi-sensor BEV splicing view lane line detection method according to claim 4, characterized in that: In step S14, the points where the integer pixel point P intersects with the center line within the step radius are the predecessor point prev and the successor point next, respectively. The coordinates (x prev ,y prev ) and the coordinates of the successor point next (x next ,y next ), the expression is: previous (x，y) =O (x，y) +α(O (x，y) -A (x，y) ) (1); next (x，y) =O (x，y) +β(B (x，y) -O (x，y) ) (2); Among them, O is the foot point of the perpendicular line from the integer pixel point P to the center line, A (x，y) , O (x，y) , B (x，y) are the coordinates of point A, point O, and point B respectively; step is the step radius of the predecessor and successor, α and β are the coefficients respectively, and l is the length of the corresponding line segment; Calculate the offset x-offset and y-offset of any integer pixel point P and its corresponding predecessor point prev and successor point next. The x-offset expression is: prev -x P , the expression of y-offset is: next -y P ; Get the predecessor regression branch reg_prev and the successor regression branch reg_next of the entire feature graph.

6. A multi-sensor BEV splicing view lane line detection method according to claim 5, characterized in that: In step S21, the solution of all predecessor points and successor points includes the following steps: S211: traverse the feature map and determine whether the current point is a point on the single lane line mask. If the probability of a single lane line is less than the threshold, skip it; if the probability of a single lane line is greater than the threshold, enter the predecessor regression branch reg_prev, obtain the offset of the current point from its corresponding predecessor point prev, and inversely calculate the precise predecessor point prev according to formula (1); S212: rounding the precise predecessor point prev, and determining whether it belongs to a point on the single lane line mask on the single lane line mask; If yes, continue to solve the predecessor point prev corresponding to the above predecessor point prev on the predecessor regression branch reg_prev until the predecessor point prev does not belong to the point on the single lane line mask, and then find all the predecessor points prev; S213: traverse the feature map and determine whether the current point is a point on the single lane line mask. If the probability of a single lane line is less than the threshold, skip it; if the probability of a single lane line is greater than the threshold, enter the subsequent regression branch reg_next, obtain the offset of the current point from its corresponding successor point next, and inversely calculate the exact successor point next according to formula (2); S212: rounding the precise successor point next, and determining whether it belongs to a point on the single lane line mask on the single lane line mask; If so, continue to solve the successor point next corresponding to the above successor point next on the successor regression branch reg_next until the successor point next does not belong to the point on the single lane line mask, and then find all the successor points next.

7. A multi-sensor BEV stitching view lane line detection method according to any one of claims 1 to 6, characterized in that: The method further includes a double lane line detection method, the double lane line detection method comprising double lane line model encoding and double lane line model decoding; The dual lane line model encoding includes the following steps: S31: distinguishing any two-lane line into a left line and a right line according to the positional relationship on the image, calculating a virtual center line of the two-lane line by the above-mentioned single-lane line detection method, and establishing a virtual center line mask; S32: traverse the virtual center line mask, encode the integer points on the virtual center line mask, and obtain each integer pixel point on the virtual center line mask; S33: by using the above single lane line detection method, the coordinates of the perpendicular points on the virtual center line mask corresponding to any integer pixel points are calculated respectively; S34: Obtain the predecessor point and the successor point corresponding to the integer pixel point through the above-mentioned single lane line detection method, and calculate the offset from the integer pixel point to the predecessor point and the successor point; S35: Draw a perpendicular line from the integer pixel point to the left line to obtain the foot point of the perpendicular from the current integer pixel point to the left line, and calculate the x-offset and y-offset from the current integer pixel point to the foot point of the perpendicular from the left line; S36: Draw a perpendicular line from the integer pixel point to the right line to obtain the foot point of the perpendicular from the current integer pixel point to the right line, and calculate the x-offset and y-offset from the current integer pixel point to the foot point of the perpendicular from the right line; S37: After traversing the virtual center line mask, the left line regression branch offset_left and the right line regression branch offset_right of the entire feature map are obtained; The two-lane line model decoding includes the following steps: S41: According to the above-mentioned single lane line detection method, complete the decoding of the virtual center line of the double lane line; S42: traverse each point on the virtual center line, and obtain the offset of the left line point and the offset of the right line point on the left line regression branch offset_left and the right line regression branch offset_right respectively according to the above single lane line detection method; S43: Using the offset in step S42, the coordinates of the left line point and the right line point are inversely calculated to obtain the entire left line and right line.

Images