A lane line detection method, device, computer equipment and storage medium

Abstract

The application relates to the technical field of lane line detection, and discloses a lane line detection method, device, computer equipment and storage medium, the method comprising the following steps: acquiring a lane image, detecting the lane image, and generating vanishing point coordinates; based on the vanishing point coordinates, a lane line detection model is used to generate a first lane line category and a second lane line category; lane line fusion is carried out based on the first lane line category and the second lane line category, and a lane line detection result is generated. The application improves the detection precision and accuracy of the lane line and reduces the safety hidden danger of automatic driving.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of lane line detection, and particularly relates to a lane line detection method and device, computer equipment and a storage medium. BACKGROUND

[0002] Lane line detection is one of the basic technologies of autonomous driving systems, and plays a crucial role in realizing the safety, stability and practicality of autonomous driving. Through accurate detection of lane lines, autonomous vehicles can determine their precise position on the road and achieve automatic lane keeping function, ensuring stable driving in the correct lane. In addition, lane line detection can also help vehicles better perceive road structure and layout, improve their perception of the surrounding environment, and thus realize higher level of environmental awareness and decision-making.

[0003] The related lane line detection method adopts a lane line detection algorithm based on key points, but this lane line detection method has potential risks of category misjudgment, which leads to certain safety hazards of autonomous driving. SUMMARY

[0004] Therefore, the present application provides a lane line detection method, device, computer equipment and storage medium to solve the potential category misjudgment problem in the lane line detection algorithm based on key points.

[0005] In a first aspect, the present application provides a lane line detection method, which comprises:

[0006] obtaining a lane image, detecting the lane image, and generating vanishing point coordinates;

[0007] generating a first lane line category and a second lane line category based on the vanishing point coordinates using a lane line detection model;

[0008] lane line fusion based on the first lane line category and the second lane line category to generate a lane line detection result.

[0009] The lane line detection method provided in this embodiment generates a lane line detection result based on lane line fusion of the first lane line category and the second lane line category, avoids category misjudgment caused by forming a lane line category according to the vehicle position, realizes multi-directional recognition of the lane line category in the lane image using the lane line detection model, improves the detection accuracy and precision of the lane line, and reduces the safety hazards of autonomous driving.

[0010] In an optional implementation, detecting the lane image to generate vanishing point coordinates comprises:

[0011] straight line extraction is performed on the lane image to generate a plurality of lane line straight lines;

[0012] The plurality of lane line straight line data is processed to generate the vanishing point coordinates.

[0013] The lane line detection method provided in the embodiment generates the vanishing point coordinates according to the lane image, so that the subsequent lane line category is determined according to the lane image instead of the position of the vehicle itself, thereby improving the robustness of the lane line detection in the lane changing scene, and the detection and recognition of the bifurcated line can be realized.

[0014] In an optional implementation, the lane image is subjected to straight line extraction to generate a plurality of lane line straight lines, including:

[0015] The lane image is subjected to grayscale processing, and the lane image subjected to the grayscale processing is subjected to Gaussian denoising processing.

[0016] The lane image subjected to the Gaussian denoising processing is subjected to edge extraction to generate a lane line image edge.

[0017] Based on the lane line edge, a mask image is used to select the lane line edge in a preset region.

[0018] Based on the lane line edge in the preset region, a Hough transform is used for straight line extraction to generate a plurality of lane line straight lines.

[0019] The lane line detection method provided in the embodiment uses the mask image to select the lane line edge in the preset region, and then uses the Hough transform for straight line extraction to generate a plurality of lane line straight lines, thereby improving the recognition precision and accuracy of the lane line straight lines.

[0020] In an optional implementation, the lane line detection model includes a backbone network, a first head network and a second head network, and the plurality of lane line straight line data is processed to generate the vanishing point coordinates, including:

[0021] The straight line start point coordinates and the straight line end point coordinates are determined based on the lane line straight line data.

[0022] The straight line matrix and the straight line vector are generated based on the straight line start point coordinates and the straight line end point coordinates.

[0023] The straight line matrix and the straight line vector are input into a straight line equation to generate the vanishing point coordinates.

[0024] The lane line detection method provided in the embodiment determines the vanishing point coordinates through the straight line start point coordinates and the straight line end point coordinates in the lane line straight line data, thereby realizing accurate recognition of the vanishing point coordinates.

[0025] In an optional implementation, the lane line detection model comprises a backbone network, a first head network and a second head network; based on the vanishing point coordinates, the lane line detection model is used to generate a first lane line category and a second lane line category, comprising:

[0026] The first head network is used to rotate and mark the plurality of lane line straight lines counterclockwise with the vanishing point coordinates as the center to generate the first lane line category;

[0027] The second head network is used to rotate and mark the plurality of lane line straight lines clockwise with the vanishing point coordinates as the center to generate the second lane line category.

[0028] The lane line detection method provided in the embodiment realizes multi-directional identification of the lane line category through the first head network and the second head network, so that the division of the lane line category is not fixed, the robustness of the lane line detection in the lane changing scene is improved, and the detection and identification of the bifurcated line can be realized.

[0029] In an optional implementation, lane line fusion is performed based on the first lane line category and the second lane line category to generate a lane line detection result, comprising:

[0030] The lane line straight line in the first lane line category is fitted with the lane line in the second lane line category to determine an error value of the lane line straight line;

[0031] The error value of the lane line straight line is compared with a preset threshold value, and based on the comparison result, the lane line straight line in the first lane line category is fused with the lane line in the second lane line category to generate the lane line detection result.

[0032] The lane line detection method provided in the embodiment determines the error value of the lane line straight line based on the lane line category, and then fuses the same lane line straight line in different lane line categories, so that the detection of the lane line is more accurate, and the risk of lane line detection error is reduced.

[0033] In an optional implementation, the lane line straight line in the first lane line category is fitted with the lane line in the second lane line category to determine the error value of the lane line straight line, comprising:

[0034] The lane line straight line corresponding to the first lane line category is randomly selected, and coordinates at the lane line straight line are randomly sampled to generate a plurality of sampling points;

[0035] The plurality of sampling points are sequentially fitted with the lane line straight line corresponding to the second lane line category to determine the error value of the lane line straight line.

[0036] The lane line detection method provided by the embodiment determines error values of lane line straight lines by sequentially performing curve fitting on the lane line straight lines corresponding to the second lane line category of the plurality of sampling points, improves the identification of the same lane line straight line in different lane line categories, reduces the identification difficulty, and makes the detection of the lane line more accurate.

[0037] In a second aspect, the present application provides a lane line detection device, which comprises:

[0038] A detection module is configured to acquire a lane image, detect the lane image, and generate vanishing point coordinates.

[0039] A generation module is configured to generate a first lane line category and a second lane line category based on the vanishing point coordinates and using a lane line detection model.

[0040] A fusion module is configured to perform lane line fusion based on the first lane line category and the second lane line category, and generate a lane line detection result.

[0041] In a third aspect, the present application provides a computer device, which comprises a memory and a processor, the memory and the processor are communicatively connected, the memory stores computer instructions, and the processor executes the computer instructions to perform the lane line detection method of the first aspect or any of the corresponding embodiments.

[0042] In a fourth aspect, the present application provides a computer readable storage medium, which stores computer instructions, and the computer instructions are used to make a computer execute the lane line detection method of the first aspect or any of the corresponding embodiments. BRIEF DESCRIPTION OF DRAWINGS

[0043] In order to more clearly illustrate the specific embodiments of the present application or the technical solutions in the prior art, the drawings needed in the specific embodiments or the prior art description will be briefly introduced as follows. Obviously, the drawings in the following description are some embodiments of the present application, and those skilled in the art can obtain other drawings according to these drawings without creative labor.

[0044] Figure 1 is a schematic diagram of lane line category division based on key points according to an embodiment of the present application;

[0045] Figure 2 is a flowchart of a lane line detection method according to an embodiment of the present application;

[0046] Figure 3 is a flowchart of another lane line detection method according to an embodiment of the present application;

[0047] Figure 4is a flowchart of still another lane line detection method according to an embodiment of the present application;

[0048] Figure 5 is a schematic diagram of lane line categories according to an embodiment of the present application;

[0049] Figure 6 is a schematic diagram of a processing procedure of a head network when lane line categories are not all 0 according to an embodiment of the present application;

[0050] Figure 7 is a flowchart of still another lane line detection method according to an embodiment of the present application;

[0051] Figure 8 is a schematic diagram of a first head network rotating a first lane line straight line to a fifth lane line straight line counterclockwise with a vanishing point coordinate as a center according to an embodiment of the present application;

[0052] Figure 9 is a schematic diagram of a second head network rotating a first lane line straight line to a fifth lane line straight line clockwise with a vanishing point coordinate as a center according to an embodiment of the present application;

[0053] Figure 10 is a schematic diagram of a point set of a lane line according to an embodiment of the present application;

[0054] Figure 11 is a schematic diagram of a lane line detection result according to an embodiment of the present application;

[0055] Figure 12 is a structural block diagram of a lane line detection device according to an embodiment of the present application;

[0056] Figure 13 is a hardware structure schematic diagram of a computer device according to an embodiment of the present application. DETAILED DESCRIPTION

[0057] To make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some but not all of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of protection of the present application.

[0058] The related lane line detection is mainly divided into two ways of re-modeling and re-post-processing. The re-post-processing scheme mostly detects lane lines through a segmentation algorithm, and then judges the structure of the lane line through complex logic of post-processing. However, the re-post-processing method needs to integrate the lane line information based on many thresholds and rules, so the industry mostly adopts the re-modeling method. The re-modeling scheme more relies on the algorithm to directly output the point position and category of the lane line. In the re-modeling algorithm, many people use the lane line detection algorithm based on key points. Because the coordinate accuracy of the algorithm based on key points is relatively high, and the reasoning speed of the algorithm is very fast, it is very suitable for deployment on the vehicle side. This kind of algorithm needs to learn fixed lane line categories. The common ones are traditional 4 categories (left one, left two, right one, right two). The division of these categories is based on the vehicle.

[0059] The lane line detection algorithm based on key points needs to divide fixed categories for learning, but this kind of algorithm basically has a potential risk of category mis-detection. Because this kind of algorithm needs to set the lane line as a fixed category, such as traditional 4 categories (left one, left two, right one, right two). Because the category division is often carried out according to certain rules, this kind of rule will fail in some situations, causing the model to mis-detect when the category division is ambiguous. For example, the method of dividing left and right categories based on the position of the vehicle. When the vehicle is changing lanes, it is difficult to determine whether the compressed lane line belongs to the left one category or the right one category. If the mis-detection is not handled well in the post-processing process, it will bring certain safety hazards to automatic driving. At the same time, this kind of algorithm cannot well detect the split line.

[0060] The related lane line detection algorithm based on key points also uses a label assignment strategy to divide categories, that is, to divide 6 lane lines of left one, left two, left three, right one, right two and right three categories, as shown in Figure 1 The specific steps are as follows: draw a vertical line in the middle of the collected image, which has an intersection with the upper boundary of the image. The lower part of the lane line in the figure is extended to have intersections with the boundaries of the image. Connecting the points and the intersections respectively obtains 6 lines. The radian values of the 6 lines and the vertical line are obtained by using the inverse tangent function to obtain θ1, θ2, θ3, θ4, θ5 and θ6. The above radian values are distinguished according to the positive and negative values and are sorted according to the size respectively. The radian values corresponding to the left 1, left 2 and left 3 lane lines on the left side of the vertical line are θ3, θ2 and θ1 respectively. The radian values corresponding to the right 1, right 2 and right 3 lane lines on the right side of the vertical line are θ4, θ5 and θ6 respectively. Thus, the corresponding 6 categories are obtained.

[0061] The lane line detection method based on the key points needs to distinguish the division of the left and right lane lines, and the division is necessarily based on the middle as the reference, but the middle is not particularly rigorous, so that the lane line placed in the middle region during the lane changing of the vehicle is blurred in the left and right categories, and thus the automatic driving has certain safety hazards.

[0062] Therefore, the embodiment of the present application provides a lane line detection method, device, computer equipment and storage medium.

[0063] According to the embodiment of the present application, a lane line detection method is provided, and it should be noted that the steps shown in the flowchart of the drawings can be executed in a computer system such as a group of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that here.

[0064] In the embodiment, a lane line detection method is provided, which can be used in the mobile terminal such as a mobile phone, a tablet computer and the like. Figure 2 The lane line detection method according to the embodiment of the present application is shown in the flowchart as shown in Figure 2 The flowchart includes the following steps:

[0065] In step S201, the lane image is acquired, the lane image is detected, and the vanishing point coordinates are generated.

[0066] Specifically, since the lane image is less, the lane line detection accuracy is insufficient, and too many lane images may cause the lane line detection accuracy to decrease, therefore, 20 or so lane images are acquired, and the 20 or so lane images are images of straight lanes.

[0067] In step S202, based on the vanishing point coordinates, a first lane line category and a second lane line category are generated by using a lane line detection model.

[0068] Specifically, the lane line detection model adopts a network structure of backbone (main network) + head (head network), wherein the main network adopts ERFNet (Efficient Residual Factorized Network, semantic segmentation convolution network), and the number of the head network can be 2, i.e. a first head network and a second head network, wherein one of the head networks is responsible for learning the semantic segmentation information of all lane lines to improve the feature extraction capability of the backbone, and the first head network and the second head network fixedly output a plurality of lane lines, for example, the first head network and the second head network fixedly output 8 lane lines.

[0069] Further, the input of the lane line detection model is the compressed image, the dimension is 1x3x256x512, the output dimension of the two heads is 1x8x256x512, and the channel number of the lane line detection model output is 8, which refers to the category, that is, 8 lane lines. The 8 lane lines here are mainly because most of the relatively wide roads are 3 lanes, 4 lane lines, plus the opposite lane, a total of 8 lane lines. The reason for setting two head networks in the lane line detection model is that if the number of lane lines is greater than 8, one head can make up for the insufficient recognition of the other head. Another reason is that if the two heads recognize the same lane line, the lane line fusion strategy will be used to combine the recognition results of the two heads to output a higher-precision lane line.

[0070] Step S203, lane line fusion based on the first lane line category and the second lane line category to generate a lane line detection result.

[0071] The lane line detection method provided in the embodiment avoids the category mis-detection caused by forming the lane line category according to the vehicle position, realizes multi-directional recognition of the lane line category in the lane image by using the lane line detection model, improves the detection precision and accuracy of the lane line, and reduces the safety hidden danger of automatic driving.

[0072] In the embodiment, a lane line detection method is provided, which can be used in the mobile terminal such as a mobile phone, a tablet computer and the like. Figure 3 The flowchart of the lane line detection method according to the embodiment of the present application is shown in FIG. 1, which includes the following steps: Figure 3

[0073] Step S301, acquiring a lane image, detecting the lane image, and generating vanishing point coordinates.

[0074] Specifically, the step S301 includes the following steps.

[0075] Step S3011, performing straight line extraction on the lane image to generate a plurality of lane line straight lines.

[0076] In some optional embodiments, the step S3011 includes the following steps.

[0077] Step a1, performing gray scale processing on the lane image, and performing Gaussian denoising processing on the lane image after the gray scale processing.

[0078] Step a2, performing edge extraction on the lane image after the Gaussian denoising processing to generate lane line image edges.

[0079] ​Specifically, the Canny operator (a multi-stage edge detection algorithm) is used to perform edge extraction on the lane image after Gaussian denoising.

[0080] In step a3, lane line edges in the preset area are selected based on the lane line edges and a mask image.

[0081] Specifically, the preset area is a triangular area, a mask image is created, and then the lane line straight lines in the triangular area are identified by using the mask image.

[0082] In step a4, straight lines are extracted based on the lane line edges in the preset area by using Hough transform, and a plurality of lane line straight lines are generated.

[0083] The lane line detection method provided in the embodiment selects lane line edges in a preset area by using a mask image, and then extracts straight lines by using Hough transform to generate a plurality of lane line straight lines, thereby improving the identification accuracy and precision of the lane line straight lines.

[0084] In step S3012, a plurality of lane line straight line data are processed to generate vanishing point coordinates.

[0085] Specifically, the least square method is used to process the plurality of lane line straight line data to generate the vanishing point coordinates.

[0086] In some optional embodiments, the above step S3012 includes:

[0087] In step b1, straight line start point coordinates and straight line end point coordinates are determined based on the lane line straight line data.

[0088] Specifically, assuming that the vanishing point coordinates are (x0, y0), the straight line start point coordinates of each lane line straight line are (x1, y1), and the straight line end point coordinates are (x2, y2).

[0089] In step b2, a straight line matrix and a straight line vector are generated based on the straight line start point coordinates and the straight line end point coordinates.

[0090] Specifically, the slope k and the intercept c of the lane line straight line are calculated based on the straight line start point coordinates (x1, y1) and the straight line end point coordinates (x2, y2), and the calculation formula is as follows:

[0091] k = (y2 - y1) / (x2 - x1) (1)

[0092] c = (x2y1 - x1y2) / (x2 - x1) (2)

[0093] Further, according to the straight line equation y0 = kx0 + c corresponding to the vanishing point, the slope k and the intercept c are brought into the straight line equation corresponding to the vanishing point to obtain the following calculation formula:

[0094] (y2-y1)x0+(x1-x2)y0=x1y2-x2y1 (3)

[0095] Further, a straight line matrix A is constructed according to [y2-y1, x1-x2], and x1y2-x2y1 is taken as a straight line vector b.

[0096] In step b3, the straight line matrix and the straight line vector are input into a straight line equation to generate the vanishing point coordinates.

[0097] Further, a pseudo-inverse matrix of the straight line matrix A is solved to generate the vanishing point coordinates, and the specific solving process is as follows: assuming that the straight line matrix A is an m*n matrix, m represents a row vector, n represents a column vector, and the vector b is a vector with a length of m, that is, the row vector of the straight line matrix is taken as the length of the vector b; then singular value decomposition is performed on the straight line matrix A to obtain U, D and V T , and the decomposition formula is as shown in the following formula;

[0098] A=UDV T (4)

[0099] In the above formula, U represents an m*n orthogonal matrix, D represents an n*n diagonal matrix, and V T represents an n*n orthogonal matrix.

[0100] Further, the pseudo-inverse matrix A + of the straight line matrix A is calculated, and the calculation formula is as shown in the following formula:

[0101] A + =VD + U T (5)

[0102] In the above formula, D + represents the pseudo-inverse matrix of D, that is, the reciprocal of the non-zero elements in D is taken and transposed, and U T represents the transpose of U.

[0103] Further, the least squares solution x is calculated, and the calculation formula is as shown in the following formula:

[0104] x=A + b (6)

[0105] Further, x corresponds to the optimal solution of the minimization objective function of the least squares problem, that is, the vanishing point coordinates; x represents the weight of the optimal solution obtained by the column vector of the straight line matrix A (that is, each column of A) when minimizing the objective function.

[0106] The lane line detection method provided in the embodiment realizes accurate identification of the vanishing point coordinates by determining the vanishing point coordinates from the straight line start point coordinates and the straight line end point coordinates in the lane line straight line data.

[0107] In step S302, a first lane line category and a second lane line category are generated based on the vanishing point coordinates by using a lane line detection model. For details, please refer to Figure 2 In step S202 of the embodiment shown in FIG. 2, no further elaboration is given here.

[0108] In step S303, lane line fusion is performed based on the first lane line category and the second lane line category to generate a lane line detection result. For details, please refer to Figure 2 In step S203 of the embodiment shown in FIG. 3, no further elaboration is given here.

[0109] The lane line detection method provided in this embodiment generates vanishing point coordinates according to lane images, so that the subsequent lane line categories are based on lane images rather than the position of the vehicle itself, thereby improving the robustness of lane line detection in lane changing scenarios and enabling detection and identification of bifurcated lines.

[0110] In this embodiment, a lane line detection method is provided, which can be used in the mobile terminal such as a mobile phone, a tablet computer and the like. Figure 4 The flowchart of the lane line detection method according to the embodiment of the present application is shown in FIG. 4, which includes the following steps: Figure 4

[0111] In step S401, a lane image is acquired, and the lane image is detected to generate vanishing point coordinates. For details, please refer to Figure 3 In step S301 of the embodiment shown in FIG. 4, no further elaboration is given here.

[0112] In step S402, a first lane line category and a second lane line category are generated based on the vanishing point coordinates by using a lane line detection model.

[0113] Specifically, the lane line detection model includes a backbone network, a first head network and a second head network, and the step S402 includes:

[0114] In step S4021, the first head network is used to rotate and mark a plurality of lane line straight lines counterclockwise with the vanishing point coordinates as the center to generate the first lane line category.

[0115] Specifically, after the vanishing point coordinates are acquired, a horizontal line passing through the vanishing point, i.e., the horizon, can be obtained. The horizon is introduced because the Y-shaped line will extend to the horizon rather than the vanishing point position.

[0116] In step S4022, the second head network is used to rotate and mark a plurality of lane line straight lines clockwise with the vanishing point coordinates as the center to generate the second lane line category.

[0117] ​Specifically, the lane line categories output by the first head network and the second head network are opposite categories, i.e., the order of the lane lines is opposite.

[0118] Further, as shown in Figure 5 the first head network is rotated counterclockwise with the vanishing point position as the center, and the lane lines corresponding to the five intersection points a-b-c-d-e from the first lane line to the fifth lane line are lane lines. Since the first head network fixedly outputs eight lane lines, and only five lane lines are marked, the lane line categories of the remaining lane lines are set to 0.

[0119] Further, the second head network is rotated clockwise with the vanishing point position as the center, and the lane lines corresponding to the five intersection points e-d-c-b-a from the first lane line to the fifth lane line are lane lines. Since the first head network fixedly outputs eight lane lines, and only five lane lines are marked, the lane line categories of the remaining lane lines are set to 0.

[0120] Further, for lane lines whose lane line categories are not all 0, the processing process of the head network is as shown in Figure 6 the corresponding lane line is a line with a value of 1, and the true value is processed by using a Gaussian algorithm.

[0121] Step S403, lane line fusion is performed based on the first lane line category and the second lane line category to generate a lane line detection result. For details, refer to step S303 of the embodiment shown in Figure 3 , which will not be repeated here.

[0122] The lane line detection method provided in this embodiment realizes multi-directional identification of lane line categories through the first head network and the second head network, so that the division of the lane line categories is not fixed, the robustness of lane line detection in scenarios such as lane changing is improved, and the detection and identification of bifurcated lines can be realized.

[0123] In this embodiment, a lane line detection method is provided, which can be used in the mobile terminal such as a mobile phone, a tablet computer, etc. Figure 7 The flowchart of the lane line detection method according to an embodiment of the present application is shown in Figure 7 , and the flowchart includes the following steps:

[0124] Step S701, a lane image is acquired, the lane image is detected, and a vanishing point coordinate is generated. For details, refer to step S401 of the embodiment shown in Figure 4 , which will not be repeated here.

[0125] Step S702, a first lane line category and a second lane line category are generated based on the vanishing point coordinate by using a lane line detection model. For details, refer toFigure 4 At step S402 of the illustrated embodiment, no further elaboration is provided herein.

[0126] At step S703, lane line fusion is performed based on the first lane line category and the second lane line category to generate a lane line detection result.

[0127] Specifically, the above step S703 includes:

[0128] At step S7031, a lane line straight line in the first lane line category is fitted with a lane line in the second lane line category to determine an error value of the lane line straight line.

[0129] In some optional embodiments, the above step S7031 includes:

[0130] At step c1, a lane line straight line corresponding to the first lane line category is randomly selected, and coordinates at the lane line straight line are randomly sampled to generate a plurality of sampling points.

[0131] At step c2, the plurality of sampling points are sequentially fitted with a lane line straight line corresponding to the second lane line category to determine an error value of the lane line straight line.

[0132] At step S7032, the error value of the lane line straight line is compared with a preset threshold value, and based on a comparison result, the lane line straight line in the first lane line category is fused with the lane line in the second lane line category to generate a lane line detection result.

[0133] Specifically, if there is an intersection between the lane lines in the first lane line category and the second lane line category, it is necessary to determine which lane line straight line in the two lane line categories is the same. Therefore, the sampling points of one lane line straight line in the first lane line category are input into the fitting equation corresponding to all lane line straight lines in the second lane line category to compare the distances, and the lane line straight line with the smallest distance is the same lane line straight line. Since each lane line category has a fixed order, after obtaining all matching pairs of lane line straight lines, the lane line straight lines corresponding to each matching pair are set as the same lane line, and thus the lane line detection result is obtained.

[0134] The lane line detection method provided in the embodiment determines an error value of a lane line straight line based on a lane line category, and then fuses the same lane line straight line in different lane line categories, so that the detection of the lane line is more accurate, and the risk of lane line detection error is reduced.

[0135] The specific steps of a lane line detection method will be described below through a specific embodiment.

[0136] Embodiment 1

[0137] The specific steps of a lane line detection method are as follows:

[0138] 20 lane images are acquired, lane images are detected, and vanishing point coordinates are generated;

[0139] As shown in Figure 8 , the first head network rotates the first lane line to the fifth lane line counterclockwise with the vanishing point coordinates as the center, as shown in Figure 9 , the second head network rotates the first lane line to the fifth lane line clockwise with the vanishing point coordinates as the center; wherein the number of lane lines detected in the lane picture is 8, therefore the output of the first head network and the second head network is missing three lane lines, therefore, lane lines need to be fused.

[0140] The rightmost lane line corresponding to the category in Figure 8 is selected, 3 points are randomly sampled in the lane line, and the 3 sampling points (x3, y3), (x4, y4), (x5, y5) are sequentially fitted with the lane line corresponding to the second lane line category, wherein the first-order curve equation F(x3'), F(x4'), F(x5'), F(x6'), F(x7') of the lane line corresponding to the second lane line category is respectively:

[0141] F(x3') = a3y + b3

[0142] F(x4') = a4y + b4

[0143] F(x5') = a5y + b5

[0144] F(x6') = a6y + b6

[0145] F(x7') = a7y + b7

[0146] Wherein a3, a4, a5, a6, a7 represent coefficients, b3, b4, b5, b6, b7 represent constants, and are known numbers.

[0147] The y coordinates of the 3 sampling points are substituted into the above 5 equations respectively to obtain 5 corresponding groups of 3 x coordinates; for example, the first lane line equation F(x3') = a3y + b3, after y3, y4, y5 are substituted, the corresponding outputs x3'3, x3'4, x3'5 are obtained; then the absolute value of the difference between the three outputs and the x coordinates of the origin (i.e. x3, x4, x5) is summed and averaged to obtain the error value of the lane line corresponding to the rightmost category in the first lane line category relative to the first lane line corresponding to the leftmost category in the second lane line category, and the calculation formula of the error value L3 is

[0148] L3=(|x3'3-x3|+|x3'4-x4|+|x3'5-x5|) / 3

[0149] The error value of the rightmost lane line in the first lane line category is determined by comparing it with the error values ​​of the fitted equations of other lane lines. The process of determining the error values ​​is the same as above, thus obtaining five error values: L3, L4, L5, L6, and L7.

[0150] The five error values ​​mentioned above are compared with a preset threshold of 8. If the error value is less than 8, it is determined to be the same lane line. Figure 8 The rightmost lane line is straight and Figure 9 The lane lines in the "right third" category are all straight lines, and the output is a set of points for the lane lines.

[0151] For example, such as Figure 10 As shown, for the same lane line, the points predicted by the left head are (lx0, ly0)(lx1, ly1), and the points predicted by the right head are (rx0, ry0)(rx1, ry1), and ly0 = ly1. For points on the same row, the x-coordinates need to be summed and averaged. The remaining points are output normally. Therefore, the final set of points for the lane line that needs to be output after fusion is (rx0, ry0)((rx1+lx0) / 2, ry1)(lx1, ly1).

[0152] The lane lines are merged according to the above steps to obtain the following result: Figure 11 The lane line detection results are shown.

[0153] This embodiment also provides a lane line detection device for implementing the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that performs a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0154] This embodiment provides a lane line detection device, such as Figure 12 As shown, it includes:

[0155] Detection module 1201 is used to acquire lane images, detect lane images, and generate vanishing point coordinates;

[0156] The generation module 1202 is used to generate a first lane line category and a second lane line category based on the vanishing point coordinates and using a lane line detection model.

[0157] The fusion module 1203 is used to perform lane line fusion based on the first lane line category and the second lane line category to generate lane line detection results.

[0158] In some optional embodiments, the detection module 1201 comprises:

[0159] a straight line extraction unit, configured to perform straight line extraction on the lane image to generate a plurality of lane line straight lines;

[0160] a processing unit, configured to process the plurality of lane line straight line data to generate vanishing point coordinates.

[0161] In some optional embodiments, the straight line extraction unit comprises:

[0162] a gray processing subunit, configured to perform gray processing on the lane image and perform Gaussian denoising processing on the lane image after the gray processing;

[0163] an edge extraction subunit, configured to perform edge extraction on the lane image after the Gaussian denoising processing to generate lane line image edges;

[0164] a selection subunit, configured to select lane line edges in a preset region based on the lane line edges by using a mask image;

[0165] a straight line extraction subunit, configured to perform straight line extraction based on the lane line edges in the preset region by using Hough transformation to generate the plurality of lane line straight lines.

[0166] In some optional embodiments, the processing unit comprises:

[0167] a determination subunit, configured to determine straight line start point coordinates and straight line end point coordinates based on the lane line straight line data;

[0168] a first generation subunit, configured to generate a straight line matrix and a straight line vector based on the straight line start point coordinates and the straight line end point coordinates;

[0169] a second generation subunit, configured to input the straight line matrix and the straight line vector into a straight line equation to generate the vanishing point coordinates.

[0170] In some optional embodiments, the generation module 1202 comprises:

[0171] a first marking unit, configured to perform counterclockwise rotation marking on the plurality of lane line straight lines by using a first head network with the vanishing point coordinates as the center to generate a first lane line category;

[0172] a second marking unit, configured to perform clockwise rotation marking on the plurality of lane line straight lines by using a second head network with the vanishing point coordinates as the center to generate a second lane line category.

[0173] In some optional embodiments, the fusion module 1203 comprises:

[0174] The fitting unit is configured to fit the lane line straight line in the first lane line category with the lane line in the second lane line category, and determine an error value of the lane line straight line;

[0175] The comparison unit is configured to compare the error value of the lane line straight line with a preset threshold, and fuse the lane line straight line in the first lane line category with the lane line in the second lane line category based on a comparison result, to generate a lane line detection result.

[0176] In some optional embodiments, the fitting unit comprises:

[0177] The sampling subunit is configured to randomly select the lane line straight line corresponding to the first lane line category, and randomly sample coordinates at the lane line straight line to generate a plurality of sampling points;

[0178] The fitting subunit is configured to sequentially perform curve fitting on the plurality of sampling points and the lane line straight line corresponding to the second lane line category, to determine the error value of the lane line straight line.

[0179] Further function descriptions of the above-mentioned various modules and units are the same as those of the above-mentioned corresponding embodiments, and will not be described here again.

[0180] The lane line detection device in the embodiment is presented in the form of a functional unit. The unit herein refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and a memory executing one or more software or fixed programs, and / or other devices that can provide the above-mentioned functions.

[0181] The embodiment of the present application further provides a computer device with the above-mentioned Figure 12 lane line detection device.

[0182] Please refer to Figure 13 , Figure 13 is a structural schematic diagram of a computer device provided by an optional embodiment of the present application, as shown in Figure 13As shown, the computer device includes one or more processors 10, memory 20, and interfaces 30 for external devices such as a keyboard and a mouse and a disk drive. One or more of the interfaces 30 enable a user to interact with the computer device. In some embodiments, the interface 30 also includes an input device, such as a microphone, or output device, such as a speaker. Figure 13 The processor 10 is used in the description as an example.

[0183] The processor 10 can be a central processing unit, a network processor, or a combination thereof. The processor 10 can further include a hardware chip. The hardware chip can be an application specific integrated circuit, a programmable logic device, or a combination thereof. The programmable logic device can be a complex programmable logic device, a field programmable logic device, a general array logic, or any combination thereof.

[0184] The memory 20 stores instructions that can be executed by the at least one processor 10 to cause the at least one processor 10 to perform the methods described in the above embodiments.

[0185] The memory 20 can include a program region and a data region. The program region can store an operating system, application programs required by at least one function, and the like. The data region can store data created according to the use of the computer device, and the like. In addition, the memory 20 can include a high-speed random access memory, and can further include a non-transitory memory such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid state storage device. In some alternative embodiments, the memory 20 can optionally include a memory disposed remotely from the processor 10, and these remote memories can be connected to the computer device through a network. Examples of the network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0186] The memory 20 can include a volatile memory, such as a random access memory, and can also include a non-volatile memory, such as a flash memory, a hard disk, or a solid state disk. The memory 20 can also include a combination of the above-mentioned types of memories.

[0187] The computer device also includes an input device 30 and an output device 40. The processor 10, the memory 20, the input device 30 and the output device 40 can be connected through a bus or other means, Figure 13 The bus connection is taken as an example.

[0188] The input device 30 can receive inputted digital or character information, and generate key signal input related to user settings and function control of the computer device, such as touch screen, keypad, mouse, trackpad, touchpad, pointing stick, one or more mouse buttons, trackball, joystick, etc. The output device 40 can include display device, auxiliary lighting device (e.g. LED), and tactile feedback device (e.g. vibration motor), etc. The display device includes but is not limited to liquid crystal display, light emitting diode, display and plasma display. In some optional embodiments, the display device can be a touch screen.

[0189] The embodiments of the present application also provide a computer readable storage medium, the method according to the embodiments of the present application can be implemented in hardware, firmware, or recorded in a storage medium, or stored in a remote storage medium or a non-transitory machine readable storage medium and downloaded from a local storage medium by network downloading of computer code, so that the method described herein can be processed by such software on a storage medium using a general purpose computer, a special purpose processor or programmable or special purpose hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only memory, a random access memory, a flash memory, a hard disk or a solid state disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that the computer, processor, microprocessor controller or programmable hardware includes a storage component that can store or receive software or computer code, when the software or computer code is accessed and executed by the computer, processor or hardware, the method shown in the above embodiments is implemented.

[0190] Although the embodiments of the present application are described in conjunction with the accompanying drawings, various modifications and changes can be made by those skilled in the art without departing from the spirit and scope of the present application, and such modifications and changes fall within the scope defined by the appended claims.

Claims

1. A lane line detection method, characterized in that, The method includes: Acquire a lane image, detect the lane image, and generate vanishing point coordinates; Based on the vanishing point coordinates, a first lane line category and a second lane line category are generated using the lane line detection model. Lane line fusion is performed based on the first lane line category and the second lane line category to generate lane line detection results; The lane detection model includes a backbone network, a first head network, and a second head network; the step of generating a first lane category and a second lane category based on the vanishing point coordinates using the lane detection model includes: Centered on the vanishing point coordinates, the first head network is used to rotate and mark multiple lane lines counterclockwise to generate the first lane line category. Centered on the vanishing point coordinates, the second head network is used to rotate and mark the multiple lane lines clockwise to generate the second lane line category; The process of fusing lane lines based on the first lane line category and the second lane line category to generate lane line detection results includes: The straight lane lines in the first lane line category are fitted to the lane lines in the second lane line category to determine the error value of the straight lane lines; The error value of the straight lane line is compared with a preset threshold. Based on the comparison result, the straight lane lines in the first lane line category are fused with the lane lines in the second lane line category to generate the lane line detection result.

2. The method according to claim 1, characterized in that, The step of detecting the lane image and generating vanishing point coordinates includes: The lane image is processed to extract straight lines, generating multiple lane lines. The straight line data of the multiple lane lines are processed to generate the coordinates of the vanishing point.

3. The method according to claim 2, characterized in that, The step of extracting straight lines from the lane image to generate multiple lane lines includes: The lane image is converted to grayscale, and the grayscale lane image is then subjected to Gaussian denoising. The lane image after Gaussian denoising is subjected to edge extraction to generate lane line image edges; Based on the lane line image edges, lane line edges within a preset area are selected using a mask image; Based on the lane line edges within the preset area, straight lines are extracted using Hough transform to generate the multiple lane line straight lines.

4. The method according to claim 3, characterized in that, The process of processing the straight-line data of the multiple lane lines to generate the vanishing point coordinates includes: The coordinates of the starting point and ending point of the straight line are determined based on the lane line straight line data. Generate a line matrix and a line vector based on the coordinates of the starting point and ending point of the line; The vanishing point coordinates are generated by inputting the linear matrix and the linear vector into the linear equation.

5. The method according to claim 1, characterized in that, The step of fitting the straight lane lines in the first lane line category with the lane lines in the second lane line category to determine the error value of the straight lane lines includes: Arbitrarily select a straight lane line corresponding to the first lane line category, and randomly sample the coordinates of the straight lane line to generate multiple sampling points; The multiple sampling points are sequentially fitted with the straight lines of the lane lines corresponding to the second lane line category to determine the error value of the straight lines of the lane lines.

6. A lane line detection device, characterized in that, The device includes: The detection module is used to acquire lane images, detect the lane images, and generate vanishing point coordinates; The generation module is used to generate a first lane line category and a second lane line category based on the vanishing point coordinates using a lane line detection model. The fusion module is used to perform lane line fusion based on the first lane line category and the second lane line category to generate lane line detection results; The generation module includes: The first marking unit is used to mark multiple lane line straight lines counterclockwise using the first head network with the vanishing point coordinates as the center, and generate the first lane line category. The second marking unit is used to mark multiple lane line straight lines clockwise with the vanishing point coordinates as the center, and to generate a second lane line category by using the second head network. The fusion module includes: The fitting unit is used to fit the straight lane lines in the first lane line category with the lane lines in the second lane line category to determine the error value of the straight lane lines. The comparison unit is used to compare the error value of the straight lane line with a preset threshold, and based on the comparison result, to fuse the straight lane lines in the first lane line category with the lane lines in the second lane line category to generate the lane line detection result.

7. A computer device, characterized in that, include: A memory and a processor are communicatively connected, the memory storing computer instructions, and the processor executing the computer instructions to perform the lane detection method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing the computer to perform the lane detection method according to any one of claims 1 to 5.

Images