A lane line detection method and system with improved color space and search window

Abstract

The present invention relates to a lane line detection method and system with improved color space and search window. Based on the perspective transformation of the lane line area image, a bird's-eye view of the lane line area is obtained, and different color components of different color spaces are used to re-integrate the bird's-eye view of the lane line area, and then the window sliding method is combined to perform lane line search and lane line fitting to achieve the final lane line detection. The scheme adopts the optimal channels in multiple color spaces for fusion, which effectively overcomes the problems of illumination, shadow, and difficulty in detecting yellow lane lines. In addition, the application of the window search method can ignore repeated calculation steps, effectively reduce the amount of calculation, and improve the speed. In addition, based on the application of mutual conversion between perspective transformation and inverse perspective transformation, a large amount of calculation involved in lane line detection is applied to an image with less data after perspective transformation, which can effectively reduce the amount of calculation, improve the detection speed of a single frame image, and meet the requirements of real-time detection.

Description

Technical Field

[0001] The invention relates to a lane line detection method and system with improved color space and search window, belonging to the technical field of computer vision. Background Art

[0002] At present, there are many methods for lane line detection. Jamel Baili et al. proposed a feature-based lane line detection method, which simplified the edge detection process by using a horizontal differential filter and grouped the detected edge points into straight lines by using an improved Hough transform. Qian Jide et al. used the Soberx edge detection operator to detect lane line edge information, and then used an improved Hough transform to detect possible lane lines based on the region of interest. Raja Muthalagu proposed that for subsequent continuous frames where lane lines are detected, lane lines must exist, and the detection of the next frame can be skipped. The above lane line detection methods are easily affected by illumination, road shadows and lane line defects, cannot detect curved lane lines, have difficulty coping with complex urban roads, are easily affected by other edge noises, and have a slow detection speed. Summary of the invention

[0003] The technical problem to be solved by the present invention is to provide a lane line detection method with improved color space and search window, which adopts a new design logic and is based on the video capture of the road ahead of the vehicle, so as to efficiently realize the detection of lane lines in the picture frame.

[0004] In order to solve the above technical problems, the present invention adopts the following technical solutions: the present invention designs a lane line detection method with improved color space and search window, based on the video capture of the road ahead of the vehicle, for the picture frame in the captured video, the following steps A to H are performed to realize the detection of the lane line in the picture frame;

[0005] Step A. Taking the picture frame as the original picture frame, performing distortion calibration on the original picture frame to obtain a first-level picture frame, and then proceeding to step B;

[0006] Step B. Obtain the lane line area image located in front of the vehicle in the primary picture frame, and apply perspective transformation to obtain a bird's-eye view of the lane line area image corresponding to the original picture frame size as the lane line area bird's-eye view, and then proceed to step C;

[0007] Step C. Convert the lane line area bird's-eye view into lane line area auxiliary color bird's-eye view corresponding to the three preset color spaces, and then select one channel of image data from each lane line area auxiliary color bird's-eye view for fusion to obtain a three-channel lane line area fusion bird's-eye view, and then proceed to step D;

[0008] Step D. Binarize the lane line area fusion bird's-eye view image according to the non-zero pixel values ​​in the lane line area and the zero pixel values ​​in other areas to obtain the lane line area fusion binary bird's-eye view image, and count the pixel distribution in the lane line area fusion binary bird's-eye view image, and obtain the image distribution histogram corresponding to the lane line area fusion binary bird's-eye view image according to the horizontal coordinate being each position on the horizontal width of the image and the vertical coordinate being the pixel value, and then proceed to step E;

[0009] Step E. According to the image distribution histogram corresponding to the lane line area fusion binary bird's-eye view, the pixel area positions of the two lane lines in the lane line area fusion binary bird's-eye view along the horizontal width direction of the image are obtained, and then the process proceeds to step F;

[0010] Step F. Based on the two-dimensional coordinate system where the lane line area fusion binary bird's-eye view is located, for two pixel area positions along the horizontal width direction of the image in the lane line area fusion binary bird's-eye view, a window with a horizontal length greater than the width of the pixel area position and a vertical width less than the minimum distance between adjacent non-zero pixel value areas in the lane line area fusion binary bird's-eye view along the vertical direction is applied, starting from the pixel area position at the bottom edge of the lane line area fusion binary bird's-eye view along the vertical upward direction, according to a preset value less than the minimum length of each non-zero pixel value area in the lane line area fusion binary bird's-eye view along the vertical direction as an interval, and the coordinate position of each pixel point of the non-zero pixel value contained in the window at each position is obtained, and the coordinate position of each pixel point is added to the array corresponding to the pixel area position; thereby obtaining the arrays corresponding to the two pixel area positions along the horizontal width direction of the image in the lane line area fusion binary bird's-eye view, and then entering step G;

[0011] Step G. For two pixel area positions along the horizontal width direction of the image in the lane line area fusion binary bird's-eye view, obtain the pixel point coordinate position of the midpoint position along the horizontal axis direction of each pixel point coordinate position corresponding to each ordinate value in the array corresponding to the pixel area position, as the pixel point coordinate position to be fitted corresponding to each ordinate value in the array; then obtain the pixel point coordinate position to be fitted corresponding to the two pixel area positions, and then enter step H;

[0012] Step H. For two pixel area positions along the horizontal width direction of the image in the lane line area fusion binary bird's-eye view, fit the coordinate positions of each pixel point to be fitted corresponding to the pixel area position to obtain the fitting equation corresponding to the pixel area position, and obtain the trajectory line in the lane line area fusion binary bird's-eye view corresponding to the fitting equation; then obtain the trajectory lines in the lane line area fusion binary bird's-eye view corresponding to the two pixel area positions, respectively, and mark them in the lane line area fusion binary bird's-eye view, and then perform inverse perspective transformation on the lane line area fusion binary bird's-eye view according to the perspective transformation operation in step B, and display the two trajectory lines in the lane line area fusion binary bird's-eye view on the primary picture frame, that is, realize the detection of the lane line in the picture frame.

[0013] As a preferred technical solution of the present invention: in the step A, firstly, an image capture device is used to capture a preset number of chessboard images at different angles; then, the Opencv library is used to perform distortion correction on the chessboard images, wherein, firstly, a grayscale image corresponding to each chessboard image is obtained, and the chessboard corners are found by the findChessboardCorners() function, and then the image capture device is calibrated by the calibrateCamera() function, and the image capture device matrix and distortion coefficient are calculated; finally, the original picture frame is calibrated for distortion according to the image capture device matrix and distortion coefficient to obtain a first-level picture frame.

[0014] As a preferred technical solution of the present invention: the step B includes the following steps B1-1 to B1-2;

[0015] Step B1-1. Based on the plane area where the primary picture frame is located, take the midpoint of the bottom edge of the primary picture frame as the starting point, extend to both sides along the straight line where the bottom edge is located by a preset distance greater than half the length of the bottom edge of the primary picture frame, obtain two bottom edge positioning points located outside the primary picture frame, and obtain the distance of the bottom edge positioning points protruding from the end point on the same side of the bottom edge along the straight line where the bottom edge is located as the protruding distance;

[0016] At the same time, starting from the midpoint of the bottom edge of the primary picture frame, extending upward along the vertical bottom edge by a preset distance less than the distance between the bottom edge of the primary picture frame and the horizon in the primary picture frame, obtaining the top edge starting point, and then extending to both sides along the straight line parallel to the bottom edge by a preset distance, respectively, obtaining two top edge positioning points located in the primary picture frame;

[0017] That is, the area image located in the quadrilateral surrounded by the two bottom edge positioning points and the two top edge positioning points in the primary picture frame is used to form the lane line area image located in front of the vehicle in the primary picture frame, and then proceed to step B1-2;

[0018] Step B1-2. Based on the original picture frame size, for the lane line area image, the two top edge positioning points of the lane line area image are respectively moved to the end points on the same side of the top edge of the original picture frame, and the two bottom edge positioning points of the lane line area image are respectively moved to the positions on the bottom edge of the original picture frame that meet the protrusion distance from the end points of the bottom edge on the same side, so as to complete the stretching of the lane line area image and realize the application of perspective transformation, so as to obtain a bird's-eye view of the lane line area image corresponding to the original picture frame size as the lane line area bird's-eye view.

[0019] As a preferred technical solution of the present invention: based on the video capture of the road ahead of the vehicle, the picture frame selected for the first time from the captured video is executed in the process of performing the steps A to H:

[0020] Step A. Taking the picture frame as the original picture frame, performing distortion calibration on the original picture frame to obtain a first-level picture frame, and then proceeding to step B;

[0021] Step B. Using the designated quadrilateral region in the primary picture frame as the lane line region image, and moving each vertex of the lane line region image according to the designated vertex positions on the top and bottom edges of the original picture frame to stretch the lane line region image, obtaining a bird's-eye view of the lane line region image corresponding to the size of the original picture frame as the lane line region bird's-eye view, and obtaining a perspective transformation matrix between the lane line region image and the lane line region bird's-eye view, and then proceeding to step C;

[0022] The second time and each subsequent time the selected picture frames from the captured video are respectively executed in the process of step A to step H:

[0023] Step A. Taking the picture frame as the original picture frame, performing distortion calibration on the original picture frame to obtain a first-level picture frame, and then proceeding to step B;

[0024] Step B. Apply the perspective transformation matrix to perform perspective transformation on the first-level picture frame to obtain a bird's-eye view of the lane line area image in the first-level picture frame corresponding to the original picture frame size as the lane line area bird's-eye view, and then proceed to step C.

[0025] As a preferred technical solution of the present invention: in the step C, image data of one channel at different sequence positions are selected from the auxiliary color bird's-eye view of each lane line area to be fused to obtain a fused three-channel lane line area fused bird's-eye view.

[0026] As a preferred technical solution of the present invention: in the step C, the lane line area bird's-eye view is first converted into a lane line area auxiliary color bird's-eye view corresponding to the HSV color space, a lane line area auxiliary color bird's-eye view corresponding to the LAB color space, and a lane line area auxiliary color bird's-eye view corresponding to the YUV color space;

[0027] Then, the image data of the V channel is selected from the auxiliary color bird's-eye view of the lane line area corresponding to the HSV color space, the image data of the A channel is selected from the auxiliary color bird's-eye view of the lane line area corresponding to the LAB color space, and the image data of the Y channel is selected from the auxiliary color bird's-eye view of the lane line area corresponding to the YUV color space;

[0028] Finally, the V channel is used as the first channel, the A channel is used as the second channel, and the Y channel is used as the third channel. The V channel image data, the A channel image data, and the Y channel image data are fused to obtain a fused bird's-eye view of the lane line area of ​​the VAY channel, and then enter step D.

[0029] As a preferred technical solution of the present invention: in the step D, the lane line area fusion bird's-eye view is subjected to grayscale processing and filtering processing in sequence, and then the lane line area fusion bird's-eye view is subjected to binarization processing to obtain a lane line area fusion binary bird's-eye view.

[0030] As a preferred technical solution of the present invention: in the step H, a quadratic polynomial fitting is performed for the coordinate position of each pixel point to be fitted corresponding to the pixel area position to obtain the quadratic polynomial fitting equation corresponding to the pixel area position, and the trajectory line in the lane line area fused binary bird's-eye view corresponding to the quadratic polynomial fitting equation is obtained.

[0031] The technical problem to be solved by the present invention is to provide a system for lane line detection method with improved color space and search window, which adopts a new design logic and can efficiently realize the detection of lane lines in picture frames based on video capture of the road ahead of the vehicle.

[0032] In order to solve the above technical problems, the present invention adopts the following technical solutions: The present invention designs a system for improving the lane line detection method of color space and search window, based on the video capture of the road ahead of the vehicle, for the captured video, each picture frame is selected in turn according to the preset interval picture frame number, and the steps A to H are respectively executed in turn to realize the detection of the lane line in the picture frame.

[0033] The lane line detection method and system with improved color space and search window described in the present invention adopts the above technical solution and has the following technical effects compared with the prior art:

[0034] The present invention designs a lane line detection method and system with improved color space and search window. Based on the perspective transformation of the lane line area image in the obtained picture frame, a bird's-eye view of the lane line area is obtained, and different color components of different color spaces are used to re-integrate the bird's-eye view of the lane line area to highlight the lane line information, and then the designed window sliding method is applied to search for lane lines, and the final lane line detection is achieved through the application of lane line fitting. In the scheme design, the optimal channels in multiple color spaces are used for fusion, which effectively overcomes the problems of illumination, shadow, and difficulty in detecting yellow lane lines. And the application of the window search method can ignore repeated calculation steps, effectively reduce the amount of calculation and improve the speed. In addition, based on the application of mutual conversion between perspective transformation and inverse perspective transformation, a large amount of calculation involved in lane line detection is applied to images with less data after perspective transformation, which can effectively reduce the amount of calculation, improve the detection speed of single-frame images, and meet the requirements of real-time detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 An overall flow chart of the lane line detection method with improved color space and search window designed for the present invention;

[0036] Figure 2 A distortion correction effect diagram provided by an embodiment of the present invention;

[0037] Figure 3 The effect diagram of selecting the ROI area range provided by the embodiment of the present invention;

[0038] Figure 4 The effect diagram after perspective transformation provided by the embodiment of the present invention;

[0039] Figure 5 A VAY color space rendering provided by an embodiment of the present invention;

[0040] Figure 6 A VAY binarization image effect diagram provided by an embodiment of the present invention;

[0041] Figure 7 A histogram statistical effect diagram of a VAY image provided by an embodiment of the present invention;

[0042] Figure 8 A schematic diagram of a sliding window search provided by an embodiment of the present invention;

[0043] Fig. 9 A rendering of lane line fitting and color marking in a bird's-eye view provided by an embodiment of the present invention;

[0044] Fig.10 The lane line detection effect provided by the embodiment of the present invention is visualized. DETAILED DESCRIPTION

[0045] The specific implementation modes of the present invention will be further described in detail below in conjunction with the accompanying drawings.

[0046] The present invention designs a lane line detection method and system with improved color space and search window, based on the video capture of the road ahead of the vehicle by a driving recorder, and selects each picture frame in turn according to the preset interval picture frame number for the captured video, such as Figure 1 As shown, the steps A to H are respectively executed in sequence to detect the lane lines in the picture frame.

[0047] Step A: Take the picture frame as the original picture frame, perform distortion calibration on the original picture frame, obtain a first-level picture frame IMG1, and then proceed to step B.

[0048] In practical applications, in the above step A, firstly, an image capture device is used to capture a preset number of chessboard images at different angles; then the Opencv library is used to perform distortion correction on the chessboard images, wherein the grayscale images corresponding to each chessboard image are first obtained, the chessboard corners are found by the findChessboardCorners() function, and then the image capture device is calibrated by the calibrateCamera() function to calculate the image capture device matrix and the distortion coefficient; finally, the original image frame is calibrated according to the image capture device matrix and the distortion coefficient to obtain a first-level image frame IMG1, as shown in FIG. Figure 2 shown.

[0049] Step B. Obtain the lane line area image IMG2 located in front of the vehicle in the first-level image frame IMG1, and apply perspective transformation to obtain a bird's-eye view of the lane line area image IMG2 corresponding to the original image frame size as the lane line area bird's-eye view IMG3, and then enter step C.

[0050] Step C. Convert the lane line area bird's-eye view IMG3 into auxiliary color bird's-eye views of the lane line area corresponding to the three preset color spaces, and then select one channel of image data at different sequence positions in each lane line area auxiliary color bird's-eye view to fuse them, and obtain the fused three-channel lane line area fused bird's-eye view IMG4, and then enter step D.

[0051] In the specific implementation application, the above step C is specifically designed to first convert the lane line area bird's-eye view IMG3 into a lane line area auxiliary color bird's-eye view corresponding to the HSV color space, a lane line area auxiliary color bird's-eye view corresponding to the LAB color space, and a lane line area auxiliary color bird's-eye view corresponding to the YUV color space;

[0052] Then, the image data of the V channel is selected from the auxiliary color bird's-eye view of the lane line area corresponding to the HSV color space, the image data of the A channel is selected from the auxiliary color bird's-eye view of the lane line area corresponding to the LAB color space, and the image data of the Y channel is selected from the auxiliary color bird's-eye view of the lane line area corresponding to the YUV color space;

[0053] Finally, the V channel is used as the first channel, the A channel is used as the second channel, and the Y channel is used as the third channel. The V channel image data, the A channel image data, and the Y channel image data are fused to obtain the fused bird's-eye view IMG4 of the lane line area of ​​the VAY channel, and then enter step D.

[0054] The experiment found that the V channel in the HSV color space, the A channel in the LAB color space, and the Y channel in the YUV color space contain the most complete raw data. Therefore, the above three channels are fused to regain the lane line area fused bird's-eye view image IMG4, which contains the most lane line data, such as Figure 5 shown.

[0055] Specifically, in the actual operation process, the conversion of RGB color space and HSV, LAB, and YUV color spaces can be used to quickly obtain the corresponding color space image. The color conversion process is real-time, and the conversion formulas of different color spaces are as follows:

[0056] The formula for converting RGB color space to HSV color space is:

[0057] C max =max(R, G, B)

[0058] C min =min(R,C,B)

[0059]

[0060] V=(max+min) / 2

[0061]

[0062] Among them, H (Hue) represents hue, S (Saturation) represents saturation, and V (Value) represents brightness.

[0063] Similarly, the corresponding Y and L component values ​​can be calculated according to the conversion formulas between RGB and YUV, and between RGB and LAB color spaces. The implementation process is to first use cvtColor() in the opencv library to convert the RGB image into a color space, then use the split() function in the opencv library to separate the converted color space, and then use the merge() function in the opencv library to reassemble the V, A, and Y components obtained after the separation into the lane line area fusion bird's-eye view IMG4.

[0064] Step D: After the lane line area fusion bird's-eye view image IMG4 is subjected to grayscale processing and filtering processing in sequence, the lane line area pixel values ​​are non-0 and the other area pixel values ​​are 0, and the lane line area fusion bird's-eye view image IMG4 is subjected to binarization processing to obtain the lane line area fusion binarization bird's-eye view image IMG5, as shown in FIG. Figure 6 As shown, the pixel distribution in the lane line area fusion binary bird's-eye view image IMG5 is counted, and the horizontal axis is the position on the horizontal width of the image and the vertical axis is the pixel value, and the image distribution histogram corresponding to the lane line area fusion binary bird's-eye view image IMG5 is obtained, as shown in Figure 7 Then go to step E.

[0065] In the specific implementation, Sobel filter is used for filtering and binarization operation to obtain IMG5, and the corresponding pixel distribution histogram is counted. Among them, Sobel filter is used to calculate the gradient amplitude and direction, and its filter convolution array is as follows: S x and S y Transverse and longitudinal filtering operators acting in the x and y directions respectively;

[0066]

[0067] Then the binary image IMG5 is obtained through thresholding operation, and its formula is as follows:

[0068]

[0069] Among them, S xy is the calculated threshold, and max() is the maximum pixel value.

[0070] Step E. According to the image distribution histogram corresponding to the lane line area fusion binary bird's-eye view image IMG5, the pixel area positions of the two lane lines in the lane line area fusion binary bird's-eye view image IMG5 along the horizontal width direction of the image are obtained, and then step F is entered.

[0071] Step F. Based on the two-dimensional coordinate system where the lane line area fusion binary bird's-eye view image IMG5 is located, for two pixel area positions along the horizontal width direction of the image in the lane line area fusion binary bird's-eye view image IMG5, a window with a horizontal length greater than the width of the pixel area position and a vertical width less than the minimum distance between adjacent non-zero pixel value areas in the lane line area fusion binary bird's-eye view image IMG5 along the vertical direction is applied, and starting from the pixel area position at the bottom edge of the lane line area fusion binary bird's-eye view image IMG5, the window moves to each position in sequence along the vertical upward direction according to a preset value less than the minimum value of the vertical length of each non-zero pixel value area in the lane line area fusion binary bird's-eye view image IMG5, such as Figure 8 As shown, the coordinate positions of each pixel point containing non-zero pixel values ​​at each position of the window are obtained, and the coordinate positions of each pixel point are added to the array corresponding to the pixel area position; then the arrays corresponding to the two pixel area positions along the horizontal width direction of the image in the lane line area fusion binary bird's-eye view image IMG5 are obtained, and then step G is entered.

[0072] Step G. For the two pixel area positions along the horizontal width direction of the lane line area fusion binary bird's-eye view image IMG5, obtain the pixel point coordinate position along the horizontal axis direction of the midpoint position of each pixel point coordinate position corresponding to each ordinate value in the array corresponding to the pixel area position, as the pixel point coordinate position to be fitted corresponding to each ordinate value in the array; then obtain the pixel point coordinate position to be fitted corresponding to the two pixel area positions, and then enter step H.

[0073] Step H. For two pixel area positions along the horizontal width direction of the image in the lane line area fusion binary bird's-eye view IMG5, quadratic polynomial fitting is performed for the coordinate positions of each pixel point to be fitted corresponding to the pixel area position, to obtain the quadratic polynomial fitting equation corresponding to the pixel area position, and obtain the trajectory line in the lane line area fusion binary bird's-eye view IMG5 corresponding to the quadratic polynomial fitting equation; then the trajectory lines in the lane line area fusion binary bird's-eye view IMG5 corresponding to the two pixel area positions are obtained, and marked in the lane line area fusion binary bird's-eye view IMG5 to form IMG6, as shown in FIG. Fig. 9 As shown, according to the perspective transformation operation in step B, an inverse perspective transformation is performed on IMG6, and the two track lines in the lane line area fusion binary bird's-eye view IMG5 are displayed on the primary image frame IMG1 to obtain IMG7, as shown in FIG. Fig.10 As shown, the lane line detection in the picture frame is realized.

[0074] In a specific implementation, the above quadratic polynomial is such as:

[0075] y=ax 2+bx+c

[0076] Where x is the horizontal coordinate of the lane line, y is the vertical coordinate corresponding to the horizontal coordinate, and a, b, and c are the corresponding polynomial coefficients. Let the width and height of IMG5 be (w, h), then the value range of x is (0, w), and the value range of y is (0, h).

[0077] Regarding the acquisition of the lane area bird's-eye view IMG3 in the above step B, in actual application, two specific implementation methods are designed to obtain it. In the first implementation method, in application, the above step B includes the following steps B1-1 to B1-2.

[0078] Step B1-1. Based on the plane area where the primary picture frame IMG1 is located, starting from the midpoint of the bottom edge of the primary picture frame IMG1 and extending to both sides along the straight line where the bottom edge is located by a preset distance greater than half the length of the bottom edge of the primary picture frame IMG1, obtain two bottom edge positioning points located outside the primary picture frame IMG1, and obtain the distance of the bottom edge positioning points protruding from the end point on the same side of the bottom edge along the straight line where the bottom edge is located as the protruding distance;

[0079] At the same time, starting from the midpoint of the bottom edge of the primary picture frame IMG1, extending upward along the vertical bottom edge by a preset distance less than the distance between the bottom edge of the primary picture frame IMG1 and the horizon of the primary picture frame IMG1, obtaining the top edge starting point, and then extending to both sides along the straight line parallel to the bottom edge by a preset distance, respectively, obtaining two top edge positioning points located in the primary picture frame IMG1;

[0080] That is, the area image located in the quadrilateral surrounded by two bottom edge positioning points and two top edge positioning points in the first-level image frame IMG1 is used to form the lane line area image IMG2 located in front of the vehicle in the first-level image frame IMG1, and then enters step B1-2.

[0081] Step B1-2. Based on the original picture frame size, for the lane line area image IMG2, the two top edge positioning points of the lane line area image IMG2 are respectively moved to the end points on the same side of the top edge of the original picture frame, and the two bottom edge positioning points of the lane line area image IMG2 are respectively moved to the positions on the bottom edge of the original picture frame that are away from the end points of the bottom edge on the same side by a protrusion distance, so as to complete the stretching of the lane line area image IMG2 and implement the perspective transformation, so as to obtain a bird's-eye view of the lane line area image IMG2 corresponding to the original picture frame size, as the lane line area bird's-eye view IMG3.

[0082] The second embodiment is based on the video capture of the road ahead of the vehicle, and the picture frame selected from the captured video for the first time during the execution of steps A to H is as follows:

[0083] Step A. Take the picture frame as the original picture frame, perform distortion calibration on the original picture frame, obtain a first-level picture frame IMG1, and then proceed to step B;

[0084] Step B. Use the designated quadrilateral area in the primary image frame IMG1 as the lane line area image IMG2, and move the vertices of the lane line area image IMG2 according to the designated vertex positions on the top and bottom edges of the original image frame to stretch the lane line area image IMG2, obtain a bird's-eye view of the lane line area image IMG2 corresponding to the original image frame size, as the lane line area bird's-eye view IMG3, and obtain the perspective transformation matrix between the lane line area image IMG2 and the lane line area bird's-eye view IMG3, and then proceed to step C.

[0085] The second time and each subsequent time the selected picture frames from the captured video are respectively executed in the process of step A to step H as follows:

[0086] Step A. Take the picture frame as the original picture frame, perform distortion calibration on the original picture frame, obtain a first-level picture frame IMG1, and then proceed to step B;

[0087] Step B. Apply the perspective transformation matrix to perform perspective transformation on the first-level picture frame IMG1 to obtain a bird's-eye view of the lane line area image IMG2 in the first-level picture frame IMG1 corresponding to the original picture frame size as the lane line area bird's-eye view IMG3, and then proceed to step C.

[0088] In the implementation example of the above step B, Figure 3 The four coordinate points corresponding to the lane line area image IMG2 are: r1(554,186), r2(721,186), r3(-89,img_h), r4(1394,img_h); Figure 4 As shown in the figure, the vertex coordinates of the four vertices after perspective transformation are: r1'(0,0), r2'(img_w,0), r3'(0,img_h), r4'(img_w,img_h), where img_w and img_h are the width and height of image IMG1 respectively; Figure 3 As shown, the coordinate points can be set on the image or outside the image. In this example, the vertices r1 and r2 are set on the image, and the vertices r3 and r4 are set outside the image. The lane line area image IMG2 is transformed into a bird's-eye view, and the lane line area bird's-eye view image IMG3 after perspective is obtained, as shown in FIG. Figure 4 As shown. And save the perspective transformation matrix M obtained in the conversion and the inverse perspective transformation matrix Minv obtained according to the inverse perspective transformation.

[0089] The above technical scheme designs a lane line detection method and system with improved color space and search window. Based on the perspective transformation of the lane line area image IMG2 in the obtained picture frame, the lane line area bird's-eye view IMG3 is obtained, and the lane line area bird's-eye view IMG3 is re-integrated using different color components of different color spaces to highlight the lane line information, and then the designed window sliding method is applied to search for lane lines, and the final lane line detection is achieved through the application of lane line fitting. In the scheme design, the optimal channels in multiple color spaces are used for fusion, which effectively overcomes the problems of illumination, shadow, and difficulty in detecting yellow lane lines. And the application of the window search method can ignore repeated calculation steps, effectively reduce the amount of calculation and improve the speed. In addition, based on the application of mutual conversion between perspective transformation and inverse perspective transformation, the large amount of calculation involved in lane line detection is applied to images with less data after perspective transformation, which can effectively reduce the amount of calculation, improve the detection speed of single-frame images, and meet the requirements of real-time detection.

[0090] Through experiments, the present invention has a good detection effect on lane lines, can accurately detect illumination, lane line defects and road surface shadows, and has good robustness. In this algorithm, the lane line boundary of a single frame image takes 0.0315 seconds, which can ensure the real-time performance of the algorithm, proving that the present invention has good application value.

[0091] The following describes the embodiments of the present invention in detail in conjunction with the accompanying drawings, but the present invention is not limited to the above embodiments, and various changes can be made within the knowledge scope of ordinary technicians in this field without departing from the purpose of the present invention.

Claims

1. A lane line detection method with improved color space and search window, characterized in that: Based on the video capture of the road ahead of the vehicle, the following steps A to H are performed for the picture frames in the captured video to detect the lane lines in the picture frames; Step A. taking the picture frame as the original picture frame, performing distortion calibration on the original picture frame to obtain a first-level picture frame, and then proceeding to step B; Step B. Obtaining the lane line area image located in front of the vehicle in the primary image frame, and applying perspective transformation to obtain a bird's-eye view of the lane line area image corresponding to the original image frame size as the lane line area bird's-eye view, and then proceeding to step C; Step C. Convert the lane line area bird's-eye view into lane line area auxiliary color bird's-eye view corresponding to the three preset color spaces, and then select one channel of image data from each lane line area auxiliary color bird's-eye view for fusion to obtain a fused three-channel lane line area fused bird's-eye view, and then proceed to step D; Step D. Binarize the lane line area fusion bird's-eye view image according to the non-zero pixel values ​​in the lane line area and the zero pixel values ​​in other areas to obtain the lane line area fusion binary bird's-eye view image, and count the pixel distribution in the lane line area fusion binary bird's-eye view image, and obtain the image distribution histogram corresponding to the lane line area fusion binary bird's-eye view image according to the horizontal coordinate being each position on the horizontal width of the image and the vertical coordinate being the pixel value, and then proceed to step E; Step E. Obtain the pixel area positions of the two lane lines in the lane line area fusion binary bird's-eye view along the horizontal width direction of the image according to the image distribution histogram corresponding to the lane line area fusion binary bird's-eye view, and then proceed to step F; Step F. Based on the two-dimensional coordinate system where the lane line area fusion binary bird's-eye view image is located, for two pixel area positions along the horizontal width direction of the image in the lane line area fusion binary bird's-eye view, a window with a horizontal length greater than the width of the pixel area position and a vertical width less than the minimum distance between adjacent non-zero pixel value areas in the lane line area fusion binary bird's-eye view along the vertical direction is applied, starting from the pixel area position at the bottom edge of the lane line area fusion binary bird's-eye view, along the vertical upward direction, according to a preset value less than the minimum length of each non-zero pixel value area in the lane line area fusion binary bird's-eye view along the vertical direction as an interval, and the coordinate position of each pixel point of the non-zero pixel value contained in the window at each position is obtained, and the coordinate position of each pixel point is added to the array corresponding to the pixel area position; thereby obtaining the arrays corresponding to the two pixel area positions along the horizontal width direction of the image in the lane line area fusion binary bird's-eye view, and then entering step G; Step G. For two pixel area positions along the horizontal width direction of the image in the lane line area fusion binary bird's-eye view, obtain the pixel point coordinate position of the midpoint position along the horizontal axis direction of each pixel point coordinate position corresponding to each ordinate value in the array corresponding to the pixel area position, as the pixel point coordinate position to be fitted corresponding to each ordinate value in the array; then obtain the pixel point coordinate position to be fitted corresponding to the two pixel area positions, and then enter step H; Step H. For two pixel area positions along the horizontal width direction of the image in the lane line area fusion binary bird's-eye view, respectively, fitting is performed for the coordinate positions of each pixel point to be fitted corresponding to the pixel area position, and the fitting equation corresponding to the pixel area position is obtained, and the trajectory line in the lane line area fusion binary bird's-eye view corresponding to the fitting equation is obtained; then the trajectory lines in the lane line area fusion binary bird's-eye view corresponding to the two pixel area positions are obtained, and marked in the lane line area fusion binary bird's-eye view, and then according to the perspective transformation operation in step B, the inverse perspective transformation is performed on the lane line area fusion binary bird's-eye view, and the two trajectory lines in the lane line area fusion binary bird's-eye view are displayed on the primary picture frame, that is, the detection of the lane line in the picture frame is realized.

2. The lane line detection method with improved color space and search window according to claim 1, characterized in that: In step A, firstly, an image capture device is used to capture a preset number of chessboard images at different angles; then, the Opencv library is used to perform distortion correction on the chessboard images, wherein, firstly, a grayscale image corresponding to each chessboard image is obtained, and the chessboard corners are found by the findChessboardCorners() function, and then the image capture device is calibrated by the calibrateCamera() function, and the image capture device matrix and distortion coefficient are calculated; finally, the original picture frame is calibrated for distortion according to the image capture device matrix and distortion coefficient to obtain a primary picture frame.

3. The lane line detection method with improved color space and search window according to claim 1, characterized in that: The step B includes the following steps B1-1 to B1-2; Step B1-1. Based on the plane area where the primary picture frame is located, take the midpoint of the bottom edge of the primary picture frame as the starting point, extend along the straight line where the bottom edge is located to both sides by a preset distance greater than half the length of the bottom edge of the primary picture frame, obtain two bottom edge positioning points located outside the primary picture frame, and obtain the distance of the bottom edge positioning points protruding from the end point on the same side of the bottom edge along the straight line where the bottom edge is located as the protruding distance; At the same time, starting from the midpoint of the bottom edge of the primary picture frame, extending upward along the vertical bottom edge by a preset distance less than the distance between the bottom edge of the primary picture frame and the horizon in the primary picture frame, obtaining the top edge starting point, and then extending to both sides along the straight line parallel to the bottom edge by a preset distance, respectively, obtaining two top edge positioning points located in the primary picture frame; That is, the area image located in the quadrilateral surrounded by the two bottom edge positioning points and the two top edge positioning points in the primary picture frame is used to form the lane line area image located in front of the vehicle in the primary picture frame, and then proceed to step B1-2; Step B1-2. Based on the original picture frame size, for the lane line area image, the two top edge positioning points of the lane line area image are respectively moved to the end points on the same side of the top edge of the original picture frame, and the two bottom edge positioning points of the lane line area image are respectively moved to the positions on the bottom edge of the original picture frame that meet the protrusion distance from the end points of the bottom edge on the same side, so as to complete the stretching of the lane line area image and realize the application of perspective transformation, so as to obtain a bird's-eye view of the lane line area image corresponding to the original picture frame size as the lane line area bird's-eye view.

4. The lane line detection method with improved color space and search window according to claim 1, characterized in that: Based on the video capture of the road ahead of the vehicle, the picture frame selected for the first time from the captured video is executed in the process of performing the steps A to H: Step A. taking the picture frame as the original picture frame, performing distortion calibration on the original picture frame to obtain a first-level picture frame, and then proceeding to step B; Step B. Using the designated quadrilateral region in the primary image frame as the lane line region image, and moving each vertex of the lane line region image according to the designated vertex positions on the top and bottom edges of the original image frame to stretch the lane line region image, obtaining a bird's-eye view of the lane line region image corresponding to the size of the original image frame as the lane line region bird's-eye view, and obtaining a perspective transformation matrix between the lane line region image and the lane line region bird's-eye view, and then proceeding to step C; The second time and each subsequent time the selected picture frames from the captured video are respectively executed in the process of step A to step H: Step A. taking the picture frame as the original picture frame, performing distortion calibration on the original picture frame to obtain a first-level picture frame, and then proceeding to step B; Step B. Apply the perspective transformation matrix to the primary picture frame to perform perspective transformation, obtain a bird's-eye view of the lane line area image in the primary picture frame corresponding to the original picture frame size as the lane line area bird's-eye view, and then proceed to step C.

5. The lane line detection method with improved color space and search window according to claim 1, characterized in that: In the step C, image data of one channel at different sequence positions are selected from the auxiliary color bird's-eye view of each lane line area respectively for fusion to obtain a fused three-channel lane line area fused bird's-eye view.

6. The lane line detection method with improved color space and search window according to claim 5, characterized in that: In the step C, the lane line area bird's-eye view is first converted into a lane line area auxiliary color bird's-eye view corresponding to the HSV color space, a lane line area auxiliary color bird's-eye view corresponding to the LAB color space, and a lane line area auxiliary color bird's-eye view corresponding to the YUV color space; Then, the image data of the V channel is selected from the auxiliary color bird's-eye view of the lane line area corresponding to the HSV color space, the image data of the A channel is selected from the auxiliary color bird's-eye view of the lane line area corresponding to the LAB color space, and the image data of the Y channel is selected from the auxiliary color bird's-eye view of the lane line area corresponding to the YUV color space; Finally, the V channel is used as the first channel, the A channel is used as the second channel, and the Y channel is used as the third channel. The V channel image data, the A channel image data, and the Y channel image data are fused to obtain a fused bird's-eye view of the lane line area of ​​the VAY channel, and then enter step D.

7. The lane line detection method with improved color space and search window according to claim 1, characterized in that: In the step D, after graying and filtering the lane line area fused bird's-eye view in sequence, the lane line area fused bird's-eye view is binarized to obtain a lane line area fused binary bird's-eye view.

8. The lane line detection method with improved color space and search window according to claim 1, characterized in that: In the step H, a quadratic polynomial is fitted for the coordinate positions of each pixel point to be fitted corresponding to the pixel area position to obtain a quadratic polynomial fitting equation corresponding to the pixel area position, and a trajectory line in the lane line area fused binary bird's-eye view corresponding to the quadratic polynomial fitting equation is obtained.

9. A system for implementing the lane line detection method with improved color space and search window as claimed in any one of claims 1 to 8, characterized in that: Based on the video capture of the road ahead of the vehicle, for the captured video, each picture frame is selected in sequence according to the preset interval picture frame number, and steps A to H are respectively executed in sequence to realize the detection of the lane lines in the picture frames.

Images