A Lane Detection Method Based on EDLines Line Features and Probability Model

Abstract

The present invention discloses a lane line detection method based on EDLines line features and a probability model. This lane line detection method uses the EDLines algorithm to extract line segments, estimates the vanishing point and uses the vanishing point to filter near-field line segments; then establishes a line segment relationship graph, performs a depth-first traversal and path checking according to the line segment relationship graph to generate candidate paths of the lane lines; then determines the candidate lane lines according to the path slope and the relative position relationship of the paths; finally, calculates the lane line parameters by using the least squares method, calculates the probability that the lane line combination is the left and right lane lines according to the probability model, and determines the lane lines according to this probability. The present invention detects line segments based on EDLines line features and determines lane lines by using a probability model, and can be realized in real time on embedded devices and mobile devices.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision, and particularly relates to a lane line detection method based on EDLines line features and a probability model. Background Art

[0002] People's living standards have been continuously improved and the economy has developed. Automobiles have gradually become a major means of transportation for every household. The ownership of automobiles is also increasing, and the construction of highways has made people's driving more convenient. However, with the increase in the number of vehicles traveling on the road, the accompanying traffic accident problems cannot be ignored. Relevant survey data shows that the main cause of traffic accidents is due to problems of the drivers themselves, and lane departure is a major cause.

[0003] The lane departure warning system is based on lane line detection, and lane line detection has certain requirements for real-time performance. The lane line detection algorithm based on deep learning has a high computational complexity. Summary of the Invention

[0004] The purpose of the present invention is to solve the above-mentioned defects in the prior art, and provide a lane line detection method based on EDLines line features and a probability model to improve the real-time performance of lane line detection.

[0005] The purpose of the present invention can be achieved by adopting the following technical solutions:

[0006] A lane line detection method based on EDLines line features and a probability model, the lane line detection method includes the following steps:

[0007] S1. Grayscale the image, use the EDLines algorithm to detect line segments from the grayscale image, and denote the obtained line segment set as AllLines. Among them, when the EDLines algorithm connects anchor points, only anchor points with the same horizontal gradient direction are connected;

[0008] S2. Vanishing point estimation, combine the vanishing point and the characteristics of the lane line to screen the candidate line segments in the near field of the current lane line, and denote the obtained line segment set as ProposalLines;

[0009] S3. Establish a line segment relationship graph, arrange all the line segments in AllLines in ascending order according to the ordinate of the starting point, establish relationships in this order, and each line segment only establishes a relationship with the line segment whose ordinate of the end point above it is less than the ordinate of the starting point of the current line segment. At the same time, use the dynamic programming method to calculate the length of the longest path for depth-first traversal with the current line segment; judge whether there is a relationship according to the included angle between two line segments, the distance from the midpoint of the relative two endpoints of the two line segments to the two line segments, and whether the horizontal gradient directions of the anchor points of the two line segments are the same;

[0010] S4. Path depth - first traversal and verification: Sort the line segments in ProposalLines in descending order according to the ordinate of the end points. Take a line segment in ProposalLines as the initial line segment in turn, and perform a traversal on the initial line segment. Start from a line segment in ProposalLines, and finally only keep the valid path with the longest length. Among them, a single traversal is a depth - first traversal and verification according to the line segment relationship graph until no next related line segment can be found. When a single traversal ends, a path starting from the initial line segment is obtained, and the fitting average error of this path is not greater than the specified threshold;

[0011] S5. Lane line screening: Divide the paths into left and right lane line edge sets according to the slope of the initial line segment of the path. Select two paths in the same lane line edge set. The distance between the intersection points of the initial line segments of the two paths and the bottom edge of the image is within the specified threshold range. The horizontal gradients of the anchor points of the line segments in the left and right paths are greater than or equal to 0 and less than or equal to 0 respectively. The two paths that meet the conditions may be the left and right edges of the same lane line and jointly form a lane line. The single path that cannot be matched forms a lane line alone; among them, the line segment slope is defined as the derivative A = du / dv, where u and v are the abscissa and ordinate of the image plane respectively, and du and dv are the abscissa change value and ordinate change value respectively;

[0012] S6. Calculate the lane line probability to determine the left and right lane lines: Divide the lane lines into left and right lane line sets according to the lane line edge set to which the path belongs. Fit the lane line parameters by the least - squares method. Calculate the probability that any pair of left and right lane line combinations are the left and right lane lines of the current lane. Select the lane line combination with the highest probability as the left and right lane lines of the current lane. Repeat the calculation process of determining the left and right lane lines of the current lane in step S6 to detect the other lane lines on the left and right sides of the current lane in turn.

[0013] Further, the process of step S2 is as follows:

[0014] S21. Use Hough transform voting to calculate the vanishing point ordinate v0. Calculate the intersection point of the extension lines of any two line segments l j and l j in AllLines, and vote on the ordinate of the intersection point. The voting score is the sum of the lengths of the line segments l i and l j . After the voting ends, select the ordinate with the highest cumulative voting score as the vanishing point ordinate v0; after obtaining the vanishing point ordinate v0, use Hough transform voting to calculate the vanishing point abscissa u0. Calculate any line segment l iThe abscissa of the intersection point between the extension line and the horizontal line with the equation v = v0, and the voting score is the length of line segment l i The length of l, and after the voting ends, the abscissa with the highest cumulative voting score is selected as the abscissa u0 of the vanishing point, and the coordinates of the obtained vanishing point V are (u0, v0); by setting the voting score weight, the robustness of the vanishing point estimation can be improved;

[0015] S22. Obtain the candidate line segments of the near-field part of the lane line: First, all line segments in AllLines whose end-point ordinates are less than the ordinate of the vanishing point are excluded; for the remaining line segments, assume that the ordinate of the starting point of line segment l i is less than that of the end point, A and B are the starting point and the end point of line segment l i respectively, V is the estimated vanishing point, and E is the midpoint of line segment l i then:

[0016]

[0017] cosθ is the cosine of the included angle formed by the vectors corresponding to line segment AB and line segment VE and spanned; when cosθ ≥ the angle comparison threshold AngleTH 1, add this line segment to the set ProposalLines; through step S22, some interfering near-field line segments that are not parallel to the lane line can be filtered out, and the interfering line segments mostly come from trees, buildings, etc. on both sides of the road.

[0018] Furthermore, in step S4, when the path performs a depth-first traversal according to the line segment relationship graph, a path check is performed on the newly added line segments; assume that the fitted lane line model is the function u = f(v), for all line segments in the path, except for the two end points of the line segment as sampling points, sample a point every sampling length Δl, fit the lane line model, and use the least squares method to solve the model parameters;

[0019] Starting from the initial line segment i, generate a path according to the line segment relationship graph. The currently generated local path is G. When traversing to line segment q, calculate the fitting average error diff,

[0020]

[0021] where n is the number of sampling points, v j is the ordinate corresponding to the j-th sampling point, and u j is the abscissa corresponding to the j-th sampling point. If diff ≤ ERRORTH, then this path passes the check, and line segment q is added to path G, otherwise this path is excluded, and the next line segment is traversed, where ERRORTH is the pre-specified comparison threshold;

[0022] Among the valid paths generated starting from the initial line segment i, the length of the longest path is CurMaxLength; when traversing to the end to generate a path, compare whether the length of the generated path is greater than CurMaxLength; if the length of the generated path is greater than CurMaxLength, update CurMaxLength to the length of the generated path and record the corresponding path.

[0023] Starting from a line segment in ProposalLines, finally only select the valid path with the longest length, the average error of the lane line model fitting of the path is not greater than the specified threshold, and connect the line segments that may belong to the edges of the same lane line.

[0024] Further, the lane line model includes the near-field slope A parameter; during the process of generating a path by depth traversal, pruning is used to reduce the search space; the near-field slope A in the lane line model is the line segment slope of the near-field line segment of the lane line. Among them, the straight line equation, or the piecewise straight line equation, or the hyperbola equation, or the polynomial equation is an optional parameter, and the present invention does not limit to a specific equation.

[0025] Further, the process of step S5 is as follows:

[0026] Connect the vanishing point and the midpoint of the bottom edge of the image, and use the slope of this line segment as the slope division threshold A for the left and right lane lines th ; the paths with the initial line segment slope less than A th belong to the left lane line edge set, otherwise they belong to the right lane line edge set, and the two paths of the structured side line belong to the same lane line edge set; where the structured side line is defined as the lane line that can detect both the left and right edges at the same time;

[0027] In the set vertical coordinate interval [v1, v2], take a horizontal line with the straight line equation v = v s at intervals of the set interval Δv in descending order of the vertical coordinate, and calculate the abscissas u1, u2 of the intersections of the two paths and the horizontal line v = v s respectively. According to the perspective transformation principle, as the vertical coordinate v s of the horizontal line gradually decreases, |u1 - u2| also decreases monotonically;

[0028] The horizontal gradients of the anchor points of the line segments in the left and right two paths are greater than or equal to 0 and less than or equal to 0 respectively; usually the lane line is brighter than the road surface, the horizontal gradient of the left edge of the lane line is greater than or equal to 0, and the horizontal gradient of the right edge of the lane line is less than or equal to 0;

[0029] For two paths that meet the above conditions, the distance between the intersection points of the initial line segment and the bottom edge of the image is distance:

[0030] distance = InterceptR - InterceptL

[0031] In the formula, InterceptL and InterceptR are respectively the abscissas of the intersection points of the initial line segments of the left and right paths and the bottom edge of the image. If LTHRESH ≤ distance ≤ HTHRESH, it is considered that the two paths are two edges constituting the structured side line, and together they form a candidate lane line. Among them, HTHRESH and LTHRESH are the pre-specified high and low comparison threshold values, and the candidate lane lines with the lane line width within a reasonable range are selected.

[0032] Further, the process of step S6 is as follows:

[0033] The lane lines are divided into left and right lane line sets according to the lane line edge set to which the paths belong. The lane line parameters are fitted by the least squares method. The probability that any pair of left and right lane line combinations is the left and right lane lines of the current lane is calculated, and the lane line combination with the maximum probability is taken as the left and right lane lines of the current lane; in the same frame of image, for different parallel lane lines, the near-field slope A in the lane line model is different, and the edge slope of the lane line is represented by the near-field slope; the difference w in the near-field slopes of parallel lane lines on the image plane is proportional to the distance d between the two lane lines on the lane plane:

[0034] w = A r -A l ∝ d

[0035] Among them, A l and A r respectively represent the near-field slopes of the left and right two parallel lane lines on the image plane;

[0036] When both sides are structured side lines, the near-field slopes of the four edges of the left and right lane lines on the image are A ll 、A lr 、A rl 、A rr from left to right, corresponding to the left edge slope of the left lane line, the right edge slope of the left lane line, the left edge slope of the right lane line, and the right edge slope of the right lane line respectively. The widths of the left and right lane lines and the lane width are d l 、d r 、d respectively. Define the vector k = [A ll A lr A rl A rr T ,D = [d l d r d] T ,The difference in the slopes of the four edges of the left and right lane lines can be represented by the vector W = [w l w r w]​T denoted as, defined as

[0037]

[0038] Since the difference in the slopes of parallel lines on the image plane is proportional to the distance on their lane plane, satisfying

[0039]

[0040] then \(M_k = 0\), where

[0041]

[0042] Through the above relationship, it is possible to determine whether a pair of lane lines in a set of parallel lines is the current lane line; due to the existence of image noise, only an estimated value of \(k\) can be obtained obeys a multivariate Gaussian distribution; defined as

[0043]

[0044] where

[0045]

[0046]

[0047]

[0048] then there is where \(E()\) is the expectation operation; under the condition that the dimensions of the known vectors \(D\) and \(D\) are independent of each other, the probability of \(\Delta\) is simplified to

[0049]

[0050] where \(D\) i is a possible combination of lane line width and lane width, \(d\) il is a possible left lane line width, \(d\) ir is a possible right lane line width, \(d\) i is a possible lane width, \(p(d\) il )、\(p(d\) ir ) and \(p(d\) i ) are the prior probabilities of the corresponding variables respectively; the larger \(p(\Delta)\) is, the greater the probability that a pair of lane line combinations corresponding to the vector is the current lane line; in the formula

[0051]

[0052] where the covariance matrix of \(\Delta\) the covariance matrix of the vector ​It is obtained by using the least squares method to solve the combined lane line model equation of multiple edges in the lane line combination; using the above probability model, select the pair of lane line combinations with the highest probability as the left and right lane lines of the current lane. Without the need for parameter calibration, it can exclude the interference of road surface arrows, etc. The road surface arrows have a parallel edge structure similar to the lane lines and are parallel to the lane lines, which are easily misjudged as lane lines.

[0053] Further, in the step S6, the unstructured lane line is defined as the lane line with only one detected edge. For the case where one side is an unstructured lane line, when the right side is an unstructured lane line, A rl = A rr , the width d of the right lane line r is fixed to 0, and Δ degenerates into

[0054] When the left side is an unstructured lane line, A ll = A lr , the width d of the left lane line l is fixed to 0, and Δ degenerates into

[0055] For the case where both sides are unstructured lane lines, use the average value of the slope differences of the left and right lane lines in the previous three frames as the discrimination criterion, and select the pair of lane lines with the closest slope difference between the left and right lane lines as the left and right lane lines of the current lane;

[0056] Due to reasons such as occlusion, shadow, and damaged lane lines, one side edge of the lane line cannot be detected. Using the above method, the left and right lane lines can still be determined using the probability model.

[0057] Further, in the step S6, repeat the calculation process of determining the left and right lane lines of the current lane to detect other lane lines; after obtaining the left and right lane lines of the current lane, use the left lane line of the current lane as the right lane line of the left lane of the current lane. According to the probability model, detect the left lane line of the left lane. Select the lane line with the highest probability in the set of left lane lines as the left lane line of the left lane. Repeat the process of detecting the left lane line of the left lane until no eligible left lane line can be found; the slope of the lane line closer to the left is smaller. The slope difference w of the lane line edges of the current lane has been obtained. The right edge slope of the left lane line of the left lane of the current lane should satisfy A rr -1.5·w ≤ A lr ≤ A rr -0.7·w, where A lr is the right edge slope of the left lane line of the left lane of the current lane, and Arr is the right edge slope of the left lane line of the current lane, which is used as the right edge slope of the right lane line of the left lane of the current lane;

[0058] Take the right lane line of the current lane as the left lane line of the right lane of the current lane. Detect the right lane line of the right lane according to the probability model, and take the lane line with the highest probability in the set of right lane lines as the right lane line of the right lane. Repeat the process of detecting the right lane line of the right lane until no eligible right lane line can be found; the slope of the lane line closer to the right is larger. The difference in the slopes of the edges of the lane lines of the current lane is w. The left edge slope of the right lane line of the right lane of the current lane should satisfy A ll +0.7·w ≤ A rl ≤ A ll +1.5·w, where A rl is the left edge slope of the right lane line of the right lane of the current lane, and A ll is the left edge slope of the right lane line of the current lane, and is used as the left edge slope of the left lane line of the right lane of the current lane;

[0059] By using the above method, multiple lane lines can be detected.

[0060] The present invention has the following advantages and effects compared with the prior art:

[0061] 1. The EDLines segment detection of the present invention only connects the anchor points with the same horizontal gradient direction, reducing false edges. By calculating the probability of the left and right lane line combinations according to the probability model, multiple lane lines can be detected. Without the need for parameter calibration, it is not easily affected by road surface arrows, etc.;

[0062] 2. The algorithm of the present invention has a low computational complexity and can be realized in real time, and is applicable to embedded devices and mobile devices. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] Figure 1 is a flowchart of the steps of the lane line detection method based on the EDLines line feature and the probability model disclosed by the present invention;

[0064] Figure 2 is a schematic diagram of judging the relationship of line segments disclosed by the present invention;

[0065] Figure 3 is a schematic diagram of the detection result of the near-field line segments of a curved lane after vanishing point screening in Embodiment 1 disclosed by the present invention;

[0066] Figure 4 is a schematic diagram of the detection result of the left and right lane lines of a curved lane in Embodiment 1 disclosed by the present invention;

[0067] Figure 5 is a schematic diagram of the detection result of the left and right lane lines of a curved lane in Embodiment 2 disclosed by the present invention;

[0068] Figure 6 is a schematic diagram of the detection result of the near-field line segments of a straight lane after vanishing point screening in Embodiment 1 disclosed by the present invention;

[0069] Figure 7 It is a schematic diagram of the detection results of the left and right lane lines in the straight lane in Embodiment 1 disclosed by the present invention;

[0070] Figure 8 It is a schematic diagram of the detection results of the left and right lane lines in the straight lane in Embodiment 2 disclosed by the present invention. Detailed implementation manners

[0071] To make the objectives, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some but not all of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0072] Embodiment 1

[0073] This embodiment discloses a lane line detection method based on EDLines line features and probability models. As Figure 1 shown, it includes the following steps:

[0074] S1. Grayscale the image, and use the EDLines algorithm to detect line segments from the grayscale image. Denote the obtained line segment set as AllLines; among them, when connecting anchor points by the EDLines algorithm, only anchor points with the same horizontal gradient direction are connected;

[0075] S2. Vanishing point estimation, combine the vanishing point and lane line characteristics to screen the candidate line segments in the near field of the lane line, and denote the obtained line segment set as ProposalLines;

[0076] S21. Use the Hough transform to vote to calculate the vertical coordinate v0 of the vanishing point. Calculate the intersection point of the extension lines of any two line segments l i and l j in AllLines, vote on the vertical coordinate of the intersection point, and the voting score is the sum of the lengths of the line segments l i and l j . After the voting ends, select the vertical coordinate with the highest cumulative voting score as the vertical coordinate v0 of the vanishing point; after obtaining the vertical coordinate v0 of the vanishing point, use the Hough transform to vote to calculate the horizontal coordinate u0 of the vanishing point, and calculate the intersection point horizontal coordinate of the extension line of any line segment l i in AllLines and the horizontal line with the straight line equation v = v0. The voting score is the line segment l iThe length. After the voting ends, select the abscissa with the highest cumulative voting score as the vanishing point abscissa u0, and the coordinates of the obtained vanishing point V are (u0, v0), where u and v are the abscissa and ordinate of the image plane respectively;

[0077] S22. Obtain candidate line segments for the near-field part of the lane line: First, exclude all line segments in AllLines whose end-point ordinates are less than the vanishing point ordinate; for the remaining line segments, assume the line segment l i The ordinate of the starting point is less than that of the end point. A and B are the starting point and end point of the line segment l i respectively, V is the estimated vanishing point, and E is the midpoint of the line segment l i Then:

[0078]

[0079] cosθ is the cosine of the included angle formed by the vectors corresponding to the line segment AB and the line segment VE and spanned; when cosθ ≥ the angle comparison threshold AngleTH 1, add this line segment to the set ProposalLines. In this embodiment, the angle threshold AngleTH 1 is taken as

[0080] As Figure 3 and Figure 6 shown, after step S2 is executed in the embodiment, most of the interfering near-field line segments not parallel to the lane line can be excluded;

[0081] S3. Establish a line segment relationship graph. Arrange all the line segments in AllLines in ascending order according to the ordinate of the starting point, and establish relationships in this order. Each line segment only establishes a relationship with the line segments whose end-point ordinates of the line segments above it are less than the ordinate of the starting point of the current line segment. At the same time, use the dynamic programming method to calculate the length of the longest path for depth-first traversal with the current line segment; when establishing a relationship between the current line segment and other line segments, the relationships of the line segments whose starting-point ordinates are less than the starting-point ordinate of the current line segment have been determined; Determine whether there is a relationship according to the included angle between the two line segments, the distances from the midpoints of the relative ends of the two line segments to the two line segments, and whether the horizontal gradient directions of the anchor points of the two line segments are the same; The specific implementation is as Figure 2 shown;

[0082] Assume that the upper and lower line segments are l i and l j respectively. The ordinate of the starting point of the line segment is less than that of the end point. I and J are the starting point and end point of the line segment l i respectively, C and D are the starting point and end point of l j respectively, and the point P is the midpoint between the point C and the point J. Then

[0083] distance = d(P, li ) + d(P, l j )

[0084]

[0085] In the above formula, d(P, l i ) and d(P, l j ) respectively represent the distances from point P to l i and l j , cosα is the cosine of the included angle formed by the vectors i and l j corresponding to l; when the horizontal gradient directions of the anchor points of line segments l and are the same and satisfy distance ≤ DistanceTH and cosα ≥ AngleTH 2, it is considered that the two lines are related, otherwise they are not related; in this embodiment, the line segment comparison threshold DistanceTH is taken as 10, and the angle comparison threshold AngleTH 2 is taken as i and l j ; According to the line segment relationship graph, starting from line segment q, perform a depth-first traversal, and the length maxscorepath of the longest path obtained

[0086] can be recursively calculated according to the following formula q :

[0087]

[0088] In the formula, L q represents the length of line segment q, graphLine q is the set of line segments related to line segment q, line segment p is the line segment in the set graphLine q , maxscorepath p is the length of the longest path obtained by performing a depth-first traversal starting from line segment p; if graphLine q is an empty set, maxscorepath q is equal to L q ;

[0089] S4. Path depth-first traversal and verification: Sort the line segments in ProposalLines in descending order according to the ordinate of the end point. Take a line segment in ProposalLines as the initial line segment in turn, and perform a traversal on the initial line segment. Starting from a line segment in ProposalLines, finally only keep the valid path with the longest length. Among them, a traversal is to perform depth-first traversal and verification according to the line segment relationship graph until no next related line segment can be found. When a traversal ends, a path starting from the initial line segment is obtained, and the fitting average error of this path is not greater than the specified threshold.

[0090] During the process of generating a path by depth traversal, pruning is used to reduce the search space.

[0091] Starting from the initial line segment i, a path is generated according to the line segment relationship graph. The currently generated partial path is G. When traversing to the line segment q, after adding the line segment q to the path G, the longest path length PossibleMaxLength that can be obtained is

[0092] PossibleMaxLength = CurLength(G) + maxscorepath q

[0093] In the formula, maxscorepath q is obtained in step S3, and CurLength(G) is the length of the current partial path G.

[0094] Among the valid paths already generated starting from the initial line segment i, the longest path length is CurMaxLength. If PossibleMaxLength is less than or equal to CurMaxLength, the line segment q is not added to the path G, and the next line segment is traversed. If PossibleMaxLength is greater than CurMaxLength, the fitting average error after adding the line segment q is verified.

[0095] When the path is performing depth-first traversal, path verification needs to be performed on the newly added line segment. In this embodiment, the fitted lane line model adopts the following hyperbolic model

[0096]

[0097] Among them, v0 is the vertical coordinate of the vanishing point, u0 is the horizontal coordinate of the vanishing point, A and B are model parameters, A is the near-field slope, and the near-field slope is the line segment slope of the near-field segment of the lane line. The line segment slope is defined as the derivative A = du / dv, where du and dv are the horizontal coordinate change value and the vertical coordinate change value respectively; for all line segments in the path, except for the two endpoints of the line segment as sampling points, a point is sampled every sampling length Δl. The sampling point data is used to fit the lane line model, and the least squares method is used to solve the model parameter vector K;

[0098] K = (X T X) -1 X T Y

[0099] In the formula, K is the model parameter vector X is a matrix Y is a vector v j is the vertical coordinate corresponding to the j-th sampling point, and u j is the horizontal coordinate corresponding to the j-th sampling point; in this embodiment, the sampling length Δl is taken as 10;

[0100] Calculate the fitting average error diff after adding the line segment q,

[0101]

[0102] Among them, the function f() is the fitted hyperbolic lane line model equation; if diff ≤ ERRORTH, the path passes the test, and the line segment q is added to the path G, otherwise the path is excluded and the next line segment is traversed; in this embodiment, the fitting error threshold ERRORTH is taken as 10;

[0103] When traversing and terminating to generate a path, compare whether the length of the generated path is greater than CurMaxLength; if the length of the generated path is greater than CurMaxLength, CurMaxLength is updated to the length of the generated path, and the corresponding path is recorded;

[0104] S5. Lane line screening: Divide the paths into left and right lane line edge sets according to the slope of the initial line segment of the path. Screen two paths in the same lane line edge set. The distance between the intersection points of the initial line segments of the two paths and the bottom edge of the image is within the specified threshold range, and the anchor point horizontal gradients of the line segments in the two paths are greater than or equal to 0 and less than or equal to 0 respectively. The two paths that meet the conditions may be the left and right edges of the same lane line and jointly form a lane line, while the single path that cannot be matched forms a lane line alone;

[0105] Connect the vanishing point and the midpoint of the bottom edge of the image, and use the slope of this line segment as the slope division threshold A of the left and right lane lines th ; The slope of the initial line segment is less than Ath The path belongs to the left lane line edge set; otherwise, it belongs to the right lane line edge set. The two paths of the structured edge line belong to the same lane line edge set. Among them, the structured edge line is defined as the lane line that can detect both the left and right edges simultaneously;

[0106] In the set ordinate interval [v1, v2], a horizontal line equation of v = v s is taken at intervals of the set interval Δv in descending order of the ordinate. The abscissa u1 and u2 of the intersection points of the path and the horizontal line v = v s are calculated respectively. As the ordinate v s of the horizontal line gradually decreases, |u1 - u2| should also decrease monotonically. In this embodiment, v1 takes the value of v2 takes the value of h, where h is the image height, and Δv takes 10;

[0107] The horizontal gradients of the anchor points of the line segments in the left and right paths are greater than or equal to 0 and less than or equal to 0 respectively;

[0108] For two paths that meet the above conditions, the distance between the intersection points of their initial line segments and the bottom edge of the image is distance:

[0109] distance = InterceptR - InterceptL

[0110] In the formula, InterceptL and InterceptR are the abscissas of the intersection points of the initial line segments of the left and right paths and the bottom edge of the image respectively. If LTHRESH ≤ distance ≤ HTHRESH, the two paths are considered as the two edges constituting the structured edge line and jointly form a candidate lane line. In this embodiment, the low comparison threshold LTHRESH is taken as 20, and the high comparison threshold HTHRESH is taken as 60;

[0111] S6. Calculate the lane line probability to determine the left and right lane lines; divide the lane lines into left and right lane line sets according to the lane line edge set to which the path belongs, fit the lane line parameters by the least squares method, calculate the probability that any pair of left and right lane line combinations is the left and right lane lines of the current lane, and take the lane line combination with the maximum probability as the left and right lane lines of the current lane;

[0112] In the same frame of image, for parallel different lane lines, the near - field slope A in the hyperbolic lane line model is different, and the edge slope of the lane line is represented by the near - field slope. When both sides are structured edge lines, A ll 、A lr 、A rl 、A rt, corresponding to the left edge slope of the left lane line, the right edge slope of the left lane line, the left edge slope of the right lane line and the right edge slope of the right lane line, respectively. The width of the left and right lane lines and the lane width are d l ,d r , d, define the vector D=[d l d r d] T ,definition

[0113]

[0114] in,

[0115]

[0116]

[0117]

[0118]

[0119] The probability of Δ is p(Δ)=∑ i p(Δ|D i )p(D i )=∑ i p(Δ|D i )p(d il )p(d ir )p(d i )

[0120] Among them, D i is the possible lane line width and lane width combination, d il is the possible width of the left lane line, d ir is the possible width of the right lane line, d i is the possible lane width, p(d il )、p(d ir ) and p(d i ) are the prior probabilities of the corresponding variables; the larger p(Δ), the larger the corresponding vector The greater the probability that a pair of lane line combinations is the current lane line; where,

[0121]

[0122] Among them, the covariance matrix of Δ Where E() is the expected operation; vector The covariance matrix of The joint lane line model equation of multiple edges in the lane line combination is solved using the least squares method.

[0123] The unstructured lane line is defined as a lane line where only one edge is detected. For the case where one side is an unstructured lane line, when the right side is an unstructured lane line, A rl = A rr , the width d of the right lane line r is fixed at 0, and Δ degenerates into

[0124] When the left side is an unstructured lane line, A ll = A lr , the width d of the left lane line l is fixed at 0, and Δ degenerates into

[0125] For the case where both sides are unstructured lane lines, the average value of the slope differences between the left and right lane lines in the previous three frames is used as the discrimination criterion. The pair of lane lines with the closest slope difference between the left and right lane lines is taken as the left and right lane lines of the current lane. In this embodiment, the possible values of the lane line widths d il and d ir are 8 cm, 10 cm, 15 cm, and 20 cm, and the possible values of the lane width d i are 3 m, 3.25 m, and 3.5 m;

[0126] Repeat the calculation process of determining the left and right lane lines of the current lane in step S6, and sequentially detect other lane lines of the left and right lanes of the current lane; after obtaining the left and right lane lines of the current lane, use the left lane line of the current lane as the right lane line of the left lane of the current lane, detect the left lane line of the left lane according to the probability model, and take the lane line with the largest probability in the set of left lane lines as the left lane line of the left lane. Repeat the process of detecting the left lane line of the left lane until no eligible left lane line can be found; the slope of the lane line closer to the left is smaller. The slope difference w between the lane line edges of the current lane has been obtained. The right edge slope A lr of the left lane line of the current lane on the left side should satisfy A rr -1.5·w ≤ A lr ≤ A rr -0.7·w, where A rr is the right edge slope of the left lane line of the current lane and serves as the right edge slope of the right lane line of the left lane of the current lane;

[0127] Use the right lane line of the current lane as the left lane line of the right lane of the current lane, detect the right lane line of the right lane according to the probability model, and take the lane line with the largest probability in the set of right lane lines as the right lane line of the right lane. Repeat the process of detecting the right lane line of the right lane until no eligible right lane line can be found; the slope of the lane line closer to the right is larger. The slope difference w between the lane line edges of the current lane has been obtained. The left edge slope A rl of the right lane line of the current lane on the right side should satisfy All +0.7·w ≤ A rl ≤ A ll +1.5·w, where A ll is the left edge slope of the right lane line of the current lane and serves as the left edge slope of the left lane line of the right lane of the current lane.

[0128] As Figure 4 shown, the fitted lane line model adopts a hyperbolic model. In Embodiment 1, multiple lane lines can be detected, including curved lane lines, unstructured side lines on the right, and two structured side lines in the middle lane;

[0129] As Figure 7 shown, in Embodiment 1, without parameter calibration, the interference of road surface arrows can be excluded. The road surface arrows are parallel to the lane lines, and the widths of both side edges are close to the width of the lane lines. Traditional methods are prone to misdetecting the two side edges of the road surface arrows as the edges of the lane lines.

[0130] Embodiment 2

[0131] This embodiment discloses a lane line detection method based on EDLines line features and probability models. As Figure 1 shown, it includes the following steps:

[0132] S1. Image grayscale processing, which can specifically refer to step S1 in Embodiment 1;

[0133] S2. Vanishing point estimation, and combining the vanishing point and lane line characteristics to screen candidate line segments in the near field of the lane line, which can specifically refer to step S2 in Embodiment 1;

[0134] S3. Establish a line segment relationship graph, which can specifically refer to step S3 in Embodiment 1;

[0135] S4. Path depth - first traversal and verification; sort the line segments in ProposalLines according to the ordinate size of the endpoints from large to small; sequentially take a line segment in ProposalLines as the initial line segment, perform a traversal on the initial line segment, start from a line segment in ProposalLines, and finally only retain the longest valid path. Among them, one traversal is to perform depth - first traversal and verification according to the line segment relationship graph until no next related line segment can be found. When one traversal ends, a path starting from the initial line segment is obtained, and the fitting average error of this path is not greater than the specified threshold;

[0136] During the process of generating a path by depth traversal, pruning is used to reduce the search space;

[0137] Generate a path starting from the initial line segment i according to the line segment relationship diagram. The currently generated partial path is G. When traversing to the line segment q, after adding the line segment q to the path G, the maximum path length PossibleMaxLength that can be obtained is

[0138] PossibleMaxLength = CurLength(G) + maxscorepath q

[0139] In the formula, maxscorepath q is obtained in step S3, and CurLength(G) is the length of the current partial path G;

[0140] Among the valid paths that have been generated starting from the initial line segment i, the maximum path length is CurMaxLength; if PossibleMaxLength is less than or equal to CurMaxLength, the line segment q is not added to the path G, and the next line segment is traversed; if PossibleMaxLength is greater than CurMaxLength, check the fitting average error after adding the line segment q;

[0141] When the path is traversed in depth - first search, path checking needs to be performed on the newly added line segment; in this embodiment, the fitted lane line model adopts the following straight - line model,

[0142] u = f(v) = Av + B

[0143] where A and B are model parameters, A is the near - field slope, and the near - field slope is the line segment slope of the near - field segment of the lane line. The line segment slope is defined as the derivative A = du / dv, where du and dv are the abscissa change value and the ordinate change value respectively; for all line segments in the path, except for the two endpoints of the line segment as sampling points, a point is sampled every sampling length Δl. The sampling point data is used to fit the lane line model, and the least - squares method is used to solve the model parameter vector K;

[0144] K = (X T X) -1 X T Y

[0145] In the formula, K is the model parameter vector X is a matrix Y is a vector v j is the ordinate corresponding to the j - th sampling point, and u j is the abscissa corresponding to the j - th sampling point; in this embodiment, the sampling length Δl is taken as 10;

[0146] Calculate the fitting average error diff after adding the line segment q,

[0147]

[0148] Among them, the function f() is the fitted straight lane line model equation; if diff ≤ ERRORTH, then this path passes the inspection, and the line segment q is added to the path G, otherwise this path is excluded and the next line segment is traversed; in this embodiment, the fitting error threshold ERRORTH is taken as 10;

[0149] When traversing to terminate and generate a path, compare whether the length of the generated path is greater than CurMaxLength; if the length of the generated path is greater than CurMaxLength, CurMaxLength is updated to the length of the generated path, and the corresponding path is recorded;

[0150] S5. Lane line screening, which can specifically refer to step S5 in Embodiment 1;

[0151] S6. Calculate the lane line probability to determine the left and right lane lines, referring to step S6 in Embodiment 1;

[0152] As Figure 5 shown, the fitted lane line model adopts a straight line model. Embodiment 2 can detect the lane lines of the near-field straight road part of the curve, and cannot accurately detect the lane lines of the far-field curve, but the computational complexity is smaller than that of the hyperbola model, and the detected near-field lane lines can still be applied to applications such as lane departure warning;

[0153] As Figure 8 shown, Embodiment 2 can exclude the interference of road surface arrows without parameter calibration, while the traditional method is prone to misdetect the two side edges of the road surface arrows as the edges of the lane lines.

[0154] The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited by the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications made without departing from the spirit and principle of the present invention shall be equivalent replacement methods and are all included in the protection scope of the present invention.

Claims

1. A lane line detection method based on EDLines line features and probability model, characterized in that The described lane line detection method includes the following steps: S1. Grayscale the image, and use the EDLines algorithm to detect line segments from the grayscale image. Denote the obtained set of line segments as AllLines. Among them, when connecting anchor points in the EDLines algorithm, only anchor points with the same horizontal gradient direction are connected; S2. Vanishing point estimation, combine the vanishing point and the characteristics of lane lines to screen the candidate line segments in the near-field part of the lane lines. Denote the obtained set of line segments as ProposalLines; S3. Establish a line segment relationship graph. Arrange all the line segments in AllLines in ascending order according to the ordinate of the starting point. Establish relationships in sequence according to this arrangement order. Each line segment only establishes a relationship with the line segment whose end point ordinate above it is less than the starting point ordinate of the current line segment. At the same time, use the dynamic programming method to calculate the length of the longest path for depth-first traversal with the current line segment; Determine whether there is a relationship according to the included angle between two line segments, the distance from the midpoint of the relative endpoints of the two line segments to the two line segments, and whether the horizontal gradient directions of the anchor points of the two line segments are the same; S4. Path depth-first traversal and verification. Sort the line segments in ProposalLines in descending order according to the ordinate of the end point. Take a line segment in ProposalLines as the initial line segment in turn, and perform a traversal on the initial line segment. Start from a line segment in ProposalLines and finally only retain the longest valid path. Among them, one traversal is to perform depth-first traversal and verification according to the line segment relationship graph until no next related line segment can be found. When one traversal ends, a path starting from the initial line segment is obtained, and the fitting average error of this path is not greater than the specified threshold; S5. Lane line screening. Divide the path into left and right lane line edge sets according to the slope of the initial line segment of the path. Screen two paths in the same lane line edge set. The distance between the intersection points of the initial line segments of the two paths and the bottom edge of the image is within the specified threshold range. The horizontal gradients of the anchor points of the line segments in the left and right paths are greater than or equal to 0 and less than or equal to 0 respectively. The two paths that meet the conditions may be the left and right edges belonging to the same lane line and jointly form a lane line. The single path that cannot be matched forms a lane line alone; Among them, the line segment slope is defined as the derivative A = du / dv, where u and v are the abscissa and ordinate of the image plane respectively, and du and dv are the abscissa change value and ordinate change value respectively; S6. Calculate the lane line probability to determine the left and right lane lines. Divide the lane lines into left and right lane line sets according to the lane line edge sets to which the paths belong. Fit the lane line parameters by the least squares method. Calculate the probability that any pair of left and right lane line combinations are the left and right lane lines of the current lane. Take the lane line combination with the maximum probability as the left and right lane lines of the current lane. Repeat the calculation process of determining the left and right lane lines of the current lane in step S6 to detect other lane lines on the left and right sides of the current lane in turn.

2. The lane line detection method based on EDLines line features and probability model according to claim 1, wherein The process of step S2 is as follows: S21. Use Hough transform voting to calculate the vertical coordinate v0 of the vanishing point. Calculate the intersection point of the extension lines of any two line segments l i and l j in AllLines, vote on the vertical coordinate of the intersection point, and the voting score is the sum of the lengths of line segments l i and l j . After the voting ends, select the vertical coordinate with the highest accumulated voting score as the vertical coordinate v0 of the vanishing point. After obtaining the vertical coordinate v0 of the vanishing point, use Hough transform voting to calculate the horizontal coordinate u0 of the vanishing point. Calculate the horizontal coordinate of the intersection point of the extension line of any line segment l i in AllLines and the horizontal line with the equation v = v0. The voting score is the length of line segment l i . After the voting ends, select the horizontal coordinate with the highest accumulated voting score as the horizontal coordinate u0 of the vanishing point. The coordinates of the obtained vanishing point V are (u0, v0); S22. Obtain candidate line segments for the near-field part of the lane line: First, exclude all line segments in AllLines whose end-point vertical coordinates are less than the vertical coordinate of the vanishing point. For the remaining line segments, let line segment l i The ordinate of the starting point is less than that of the ending point. Let A and B be the starting point and the ending point of line segment l respectively i V is the estimated vanishing point, and E is the midpoint of line segment l i Then: cosθ is the cosine of the included angle formed by the vectors corresponding to line segment AB and line segment VE and When cosθ ≥ the angle comparison threshold AngleTH 1, add this line segment to the set ProposalLines.

3. A lane line detection method based on EDLines line features and probability model according to claim 1, characterized in that In step S4, when the path performs a depth-first traversal according to the line segment relationship graph, path inspection is performed on the newly added line segments. Assume that the fitted lane line model is the function u = f(v). For all line segments in the path, except for the two end points of the line segment as sampling points, sample a point every sampling length Δl, fit the lane line model, and use the least squares method to solve the model parameters. Starting from the initial line segment i, generate a path according to the line segment relationship graph. The currently generated local path is G. When traversing to the line segment q, calculate the fitting average error diff. where n is the number of sampling points, v j is the ordinate corresponding to the j-th sampling point, u j is the abscissa corresponding to the j-th sampling point. If diff ≤ ERRORTH, then this path passes the test, and the line segment q is added to the path G; otherwise, this path is excluded and the next line segment is traversed, where ERRORTH is a pre-specified comparison threshold; Among the valid paths that have been generated starting from the initial line segment i, the length of the longest path is CurMaxLength. When a path is generated at the end of the traversal, compare whether the length of the generated path is greater than CurMaxLength. If the length of the generated path is greater than CurMaxLength, update CurMaxLength to the length of the generated path and record the corresponding path. Starting from a line segment in ProposalLines, finally only select the valid path with the longest length.

4. A lane line detection method based on EDLines line features and probability model according to claim 3, characterized in that, The lane line model contains the near-field slope A parameter. During the process of generating a path by depth traversal, pruning is used to reduce the search space. The near-field slope A in the lane line model is the line segment slope of the near-field line segment of the lane line.

5. A lane line detection method based on EDLines line features and probability model according to claim 1, characterized in that, The process of step S5 is as follows: Connect the vanishing point and the midpoint of the bottom edge of the image, and use the slope of this line segment as the threshold A for dividing the slopes of the left and right lane lines. th ; If the slope of the initial line segment is less than A th the path belongs to the left lane line edge set, otherwise it belongs to the right lane line edge set. The two paths of the structured edge line belong to the same lane line edge set; Among them, the structured edge line is defined as the lane line that can detect both the left and right edges at the same time. In the set vertical coordinate interval [v1, v2], a horizontal line with the equation v = v is taken at intervals of Δv in descending order of the vertical coordinate. s The abscissas u1 and u2 of the intersections of the two paths with the horizontal line v = v are calculated respectively. s As the vertical coordinate v of the horizontal line s gradually decreases, |u1 - u2| also decreases monotonically. The horizontal gradients of the anchor points of the line segments in the left and right paths are greater than or equal to 0 and less than or equal to 0 respectively. For two paths that meet the above conditions, the distance between the intersection points of the initial line segments and the bottom edge of the image is distance: distance = InterceptR - InterceptL In the formula, InterceptL and InterceptR are the abscissas of the intersection points of the initial line segments of the left and right paths and the bottom edge of the image respectively. If HTHRESH ≤ distance ≤ HTHRESH, it is considered that the two paths are the two edges that make up the structured edge line and jointly form a candidate lane line. Among them, HTHRESH and LTHRESH are the pre-specified high and low comparison threshold values.

6. The lane line detection method based on EDLines line features and probability model according to claim 1, characterized in that The process of step S6 is as follows: The lane lines are divided into left and right lane line sets according to the edge set of the lane lines to which the path belongs. The lane line parameters are fitted by the least squares method. The probability that any pair of left and right lane lines is the left and right lane lines of the current lane is calculated according to the lane line parameters, and the lane line combination with the maximum probability is taken as the left and right lane lines of the current lane. In the same frame of image, for different parallel lane lines, the near-field slope A in the lane line model is different, and the edge slope of the lane line is represented by the near-field slope. When both sides are structured side lines, the near-field slopes of the four edges of the left and right lane lines on the image are A ll , A lr , A rl , A rr , corresponding to the left edge slope of the left lane line, the right edge slope of the left lane line, the left edge slope of the right lane line, and the right edge slope of the right lane line respectively. The widths of the left and right lane lines of the lane and the lane width are d l , d r , d. Define the vector D = [d l d r d] T , define Among them, The probability of Δ is p(Δ) = ∑ i p(Δ|D i )p(D i ) = ∑ i p(Δ|D i )p(d il )p(d ir )p(d i ) Among them, D i is a possible combination of lane line width and lane width, d il is a possible left lane line width, d ir is a possible right lane line width, d i is a possible lane width, p(d il ), p(d ir ) and p(d i ) are the prior probabilities of the corresponding variables respectively; the larger p(Δ) is, the greater the probability that a pair of lane line combinations of the corresponding vector is the current lane line; in the formula, Among them, the covariance matrix of Δ where E() is the expectation operation; the vector covariance matrix of is obtained by solving the joint lane line model equation of multiple edges in the lane line combination using the least squares method.

7. A lane line detection method based on EDLines line features and probability model according to claim 6, characterized in that, In the step S6, the unstructured side line is defined as a lane line where only one edge is detected. For the case where one side is an unstructured side line, when the right side is an unstructured side line, A rl = A rr , the width d of the right lane line r is fixed to 0, and Δ degenerates into When the left side is an unstructured boundary line, A ll = A lr , the width d of the left lane line l is fixed at 0, and Δ degenerates into For the case where both sides are unstructured edge lines, use the average value of the slope differences between the left and right lane lines in the previous three frames as the discrimination criterion, and select the pair of lane lines with the closest slope difference between the left and right lane lines as the left and right lane lines of the current lane.

8. A lane line detection method based on EDLines line features and probability model according to claim 1, characterized in that, In the step S6, the calculation process of repeatedly determining the left and right lane lines of the current lane is performed to detect other lane lines. After obtaining the left and right lane lines of the current lane, the left lane line of the current lane is used as the right lane line of the left lane of the current lane. According to the probability model, the left lane line of the left lane is detected, and the lane line with the maximum probability in the left lane line set is taken as the left lane line of the left lane. The process of repeatedly detecting the left lane line of the left lane is performed until no eligible left lane line can be found. The slope of the lane line closer to the left is smaller. The difference in the slopes of the edges of the lane lines of the current lane, w, has been obtained. The right edge slope of the left lane line of the left lane of the current lane should satisfy A rr -1.5·w ≤ A lr ≤ A rr -0.7·w, where A lr is the right edge slope of the left lane line of the left lane of the current lane, and A rr is the right edge slope of the left lane line of the current lane and is used as the right edge slope of the right lane line of the left lane of the current lane; Use the right lane line of the current lane as the left lane line of the right lane of the current lane, detect the right lane line of the right lane according to the probability model, and select the lane line with the largest probability in the set of right lane lines as the right lane line of the right lane. Repeat the process of detecting the right lane line of the right lane until no eligible right lane line can be found. The slope of the lane line on the right is greater. The difference in the slopes of the edges of the lane lines of the current lane, w, has been obtained. The left edge slope of the right lane line of the lane on the right of the current lane should satisfy A ll +0.7·w ≤ A rl ≤ A ll +1.5·w, where A rl is the left edge slope of the right lane line of the lane on the right of the current lane, and A ll is the left edge slope of the right lane line of the current lane and serves as the left edge slope of the left lane line of the lane on the right of the current lane.

Images