Lane line detection method and device, equipment and storage medium

Abstract

The invention relates to a lane line detection method and device, equipment and a storage medium, and the method comprises the steps: carrying out the preprocessing based on a shot image, and obtaining a binary image; searching a starting pixel point and an ending pixel point of the lane line from a plurality of pixel points to be detected included in the binary image; based on the starting pixel point and the ending pixel point of the lane line and the first pixel point, under the condition that it is determined that a preset condition is met, the first pixel point serves as a middle pixel point of the lane line; wherein the first pixel point is obtained by traversing the pixel points to be detected based on a preset sequence; and taking the middle pixel point as a next starting pixel point, continuously determining whether a preset condition is met or not until the next first pixel point is a termination pixel point, and drawing all the found middle pixel points on the shot image.

Description

Technical Field

[0001] The present invention relates to the field of lane line detection for intelligent driving, and in particular to a lane line detection method and device, equipment, and storage medium. Background Art

[0002] In the field of intelligent driving, intelligent driving software needs to perceive the environment around the vehicle and output the perception results. What is important is that the intelligent driving software needs to identify the lane lines around the vehicle. Currently, traditional computer vision methods are used to identify lane lines in the image domain. However, this method has high requirements for the hardware platform, resulting in increased costs. Summary of the invention

[0003] The present invention provides a lane line detection method to solve the problem of increased cost; also provides a lane line detection device; also provides an electronic device; also provides a computer-readable storage medium; and also provides a computer program product.

[0004] In order to achieve the above object, the technical solution adopted by the present invention is as follows:

[0005] A lane line detection method, characterized in that the lane line detection method comprises:

[0006] Preprocessing is performed based on the captured image to obtain a binary image;

[0007] Finding the starting pixel point and the ending pixel point of the lane line from the plurality of pixels to be tested included in the binary image;

[0008] Based on the starting pixel point and the ending pixel point of the lane line, and the first pixel point, if a preset condition is met, the first pixel point is used as the middle pixel point of the lane line; wherein the first pixel point is obtained by traversing the pixel points to be tested in a preset order;

[0009] The intermediate pixel point is used as the next starting pixel point, and whether the preset condition is satisfied is continuously determined until the next first pixel point is the ending pixel point, and all the intermediate pixel points found are then drawn on the captured image.

[0010] According to the above technical means, based on the binary image corresponding to the captured image, the starting pixel point and the ending pixel point of the lane line are found, and then combined with the preset conditions, when it is determined that the first pixel point traversed in the preset order is the middle pixel point between the starting pixel point and the ending pixel point, the first pixel point is used as the next starting pixel point, and combined with the next first pixel point, the steps of whether the preset conditions are met are continued until the next first pixel point is the ending pixel point, indicating that the search process for the middle pixel point of the lane line is completed, and finally all the pixel points of the lane line are drawn on the captured image to help the owner correctly identify the lane line and ensure driving safety. In addition, this method of determining the lane line detection has low requirements on the hardware platform, thereby achieving the purpose of reducing certain costs.

[0011] Further, the preset conditions include: the ground distance between the starting pixel point and the first pixel point is less than a distance threshold, and the ground distance between the first pixel point and the ending pixel point is less than the distance threshold, and a multiple of a first preset value of the ground distance between the starting pixel point and the ending pixel point is greater than the sum of the ground distance between the starting pixel point and the first pixel point and the ground distance between the first pixel point and the ending pixel point;

[0012] The distance threshold is determined based on the ground distance between the starting pixel point and the ending pixel point.

[0013] According to the above technical means, the ground distance between the starting pixel and the first pixel is less than the distance threshold, and the ground distance between the first pixel and the ending pixel is less than the distance threshold, in order to limit the first pixel to between the starting pixel and the ending pixel. In addition, the multiple of the first preset value of the ground distance between the starting pixel and the ending pixel is greater than the sum of the ground distance between the starting pixel and the first pixel and the ground distance between the first pixel and the ending pixel, in order to allow the lane line of the curve to be found. In this way, when all three are satisfied, it means that the first pixel is an intermediate pixel between the starting pixel and the ending pixel.

[0014] Furthermore, the preset condition also includes: the absolute value of the difference between the first slope and the second slope is less than the slope threshold;

[0015] The first slope is determined based on the pixel coordinates of the starting pixel point and the pixel coordinates of the ending pixel point, and the second slope is determined based on the pixel coordinates of the starting pixel point and the pixel coordinates of the first pixel point.

[0016] According to the above technical means, when the absolute value of the difference between the first slope and the second slope is less than the slope threshold, it indicates that the starting pixel point, the ending pixel point and the first pixel point are pixels on the same lane line.

[0017] Further, the method of finding the starting pixel point and the ending pixel point of the lane line from the multiple pixel points to be measured included in the binary image includes: converting the pixel coordinates of the multiple pixel points to be measured into ground coordinates based on the internal and external parameter matrices of the camera; determining the starting ground coordinates of the lane line on one side that is closest to the vehicle based on the ground coordinates of the multiple pixel points to be measured; using the pixel point corresponding to the starting ground coordinate as the starting pixel point; determining the starting ground coordinates of the lane line on one side based on the starting ground coordinates and the lane width; determining the ending ground coordinates of each lane line based on the starting ground coordinates and the lane line detection distance; and using the pixel point corresponding to the ending ground coordinate as the ending pixel point.

[0018] According to the above technical means, the starting pixel point and the ending pixel point of the lane line can be quickly found in the above method.

[0019] Further, the method of determining the starting ground coordinates of the lane line on one side that is closest to the vehicle based on the ground coordinates of the multiple pixel points to be measured includes: selecting the ground coordinates of multiple selected pixel points located on one side of the vehicle from the ground coordinates of the multiple pixel points to be measured; determining the distance between the ground coordinates of each selected pixel point and the ground coordinates of the vehicle; and determining the starting ground coordinates of the lane line on one side that is closest to the vehicle based on the ground coordinates of the selected pixel point corresponding to the minimum distance.

[0020] Furthermore, the preprocessing is performed based on the captured image to obtain a binary image, including: cropping the captured image based on the internal and external parameter matrices of the camera and the lane line detection distance to obtain a region of interest; performing grayscale processing on the region of interest to obtain a grayscale image; and binarizing the grayscale image to obtain the binary image.

[0021] According to the above technical means, by cutting out the region of interest at a certain distance from the vehicle, and then graying and binarizing the region of interest, a binary image is obtained. This reduces the amount of calculation and thus improves the lane line detection rate.

[0022] Furthermore, the captured image is cropped based on the camera's internal and external parameter matrix and the lane line detection distance to obtain the region of interest, including: determining a first pixel coordinate based on a first ground coordinate and the camera's internal and external parameter matrix; wherein the horizontal coordinate of the first ground coordinate is zero, and the vertical coordinate is the lane line detection distance; using the vertical coordinate of the first pixel coordinate as the cropping width, and cropping the captured image from top to bottom to obtain the region of interest.

[0023] A lane line detection device, characterized in that the lane line detection device comprises:

[0024] A processing unit, used for performing preprocessing based on the captured image to obtain a binary image;

[0025] The processing unit is further used to find the starting pixel point and the ending pixel point of the lane line from the multiple pixels to be tested included in the binary image;

[0026] The processing unit is further configured to determine, based on the starting pixel point and the ending pixel point of the lane line, and the first pixel point, that the first pixel point is used as the middle pixel point of the lane line when a preset condition is met; wherein the first pixel point is obtained by traversing the pixel points to be tested in a preset order;

[0027] The processing unit is also used to use the intermediate pixel point as the next starting pixel point, and continue to determine whether the preset condition is met until the next first pixel point is the ending pixel point. The drawing unit is used to draw all the found intermediate pixel points on the captured image.

[0028] An electronic device comprises: a processor and a memory configured to store a computer program that can be run on the processor, wherein the processor is configured to execute the steps of the above method when running the computer program.

[0029] A computer-readable storage medium stores a computer program, which implements the steps of the above method when executed by a processor.

[0030] A computer program product comprises a computer program or instructions, wherein when the computer program or instructions are executed by a processor, the steps of the aforementioned method are implemented.

[0031] Beneficial effects of the present invention:

[0032] (1) The present invention searches for the starting pixel point and the ending pixel point of the lane line based on the binary image corresponding to the captured image, and then combines the preset conditions to determine that the first pixel point traversed in the preset order is the middle pixel point between the starting pixel point and the ending pixel point. The first pixel point is used as the next starting pixel point, and combined with the next first pixel point, the steps of whether the preset conditions are met are continued until the next first pixel point is the ending pixel point, indicating that the search process for the middle pixel point of the lane line is completed. Finally, all the pixel points of the lane line are drawn on the captured image to help the car owner correctly identify the lane line and ensure driving safety. In addition, this method of determining the lane line detection has low requirements on the hardware platform, thereby achieving the purpose of reducing a certain cost.

[0033] (2) In the present invention, the ground distance between the starting pixel and the first pixel is less than the distance threshold, and the ground distance between the first pixel and the ending pixel is less than the distance threshold, in order to limit the first pixel to be between the starting pixel and the ending pixel. In addition, the multiple of the first preset value of the ground distance between the starting pixel and the ending pixel is greater than the sum of the ground distance between the starting pixel and the first pixel and the ground distance between the first pixel and the ending pixel, in order to allow the lane line of the curve to be found. In this way, when all three are satisfied, it means that the first pixel is an intermediate pixel between the starting pixel and the ending pixel. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 A schematic diagram of a lane line detection method provided by an embodiment of the present invention Figure 1 ;

[0035] Figure 2 A schematic diagram of a lane line detection method provided by an embodiment of the present invention Figure 2 ;

[0036] Figure 3 A schematic diagram of a lane line detection method provided by an embodiment of the present invention Figure 3 ;

[0037] Figure 4 A schematic diagram of a lane line detection method provided by an embodiment of the present invention Figure 4 ;

[0038] Figure 5 A schematic diagram of a lane line detection method provided by an embodiment of the present invention Figure 5 ;

[0039] Figure 6 A schematic diagram of the structure of a lane line detection device provided in an embodiment of the present invention Figure 1 ;

[0040] Figure 7 A schematic diagram of a binary image provided by an embodiment of the present invention;

[0041] Figure 8 A schematic diagram of a captured image with lane line detection points marked provided by an embodiment of the present invention;

[0042] Fig. 9 A schematic diagram of the structure of a lane line detection device provided in an embodiment of the present invention Figure 2 ;

[0043] Fig.10 A schematic diagram of the composition structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0044] In order to enable a more detailed understanding of the features and technical contents of the embodiments of the present invention, the implementation of the embodiments of the present invention is described in detail below in conjunction with the accompanying drawings. The attached drawings are for reference only and are not intended to limit the embodiments of the present invention.

[0045] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used herein are only for the purpose of describing the present embodiment and are not intended to limit the present invention.

[0046] In the following description, references to “some embodiments,” “this embodiment,” “this embodiment,” and examples, etc., describe a subset of all possible embodiments, but it can be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments, and may be combined with each other without conflict.

[0047] If similar descriptions of "first / second" appear in the application documents, the following instructions are added. In the following description, the terms "first\second\third" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged in a specific order or sequence where permitted, so that the present embodiment described here can be implemented in an order other than that illustrated or described here.

[0048] In this embodiment, the term "and / or" is merely a description of the association relationship between associated objects, indicating that three relationships may exist. For example, object A and / or object B may represent three situations: object A exists alone, object A and object B exist at the same time, and object B exists alone.

[0049] The embodiment of the present invention provides a lane line detection method. Figure 1 A schematic diagram of a lane line detection method provided by an embodiment of the present invention Figure 1 , used in electronic devices such as Figure 1 As shown, the lane line detection method includes the following steps:

[0050] Step S101: pre-processing is performed based on the captured image to obtain a binary image.

[0051] In the embodiment of the present invention, the captured images included in the multimedia files of the camera or hard disk are stored in a preset storage space in frame order, and then the captured images are preprocessed in sequence to obtain binary images. The captured images are images including the left and right lane lines of the vehicle.

[0052] In the embodiment of the present invention, the captured image is gray-processed to obtain a gray-scale image, and the gray-scale image is binarized to obtain a binary image.

[0053] It should be noted that the formats of the captured images include but are not limited to RGB format, RAW format, and YUV format. A binary image, also known as a binary image or a black-and-white image, is an image that contains only two pixel values ​​(0 and 255). A pixel value of 0 represents black, and a pixel value of 255 represents white.

[0054] Step S102: Find the starting pixel point and the ending pixel point of the lane line from the multiple pixels to be tested included in the binary image.

[0055] In the embodiment of the present invention, the pixel point to be tested refers to a pixel point with a pixel value of 255.

[0056] In an embodiment of the present invention, the lane lines include a left lane line and a right lane line. Therefore, the starting pixel point and the ending pixel point of the left lane line, as well as the starting pixel point and the ending pixel point of the right lane line are found from the multiple pixel points to be tested included in the binary image.

[0057] Step S103: Based on the starting pixel point and the ending pixel point of the lane line, and the first pixel point, when it is determined that the preset conditions are met, the first pixel point is used as the middle pixel point of the lane line; wherein the first pixel point is obtained by traversing the pixel points to be tested based on a preset order.

[0058] In an embodiment of the present invention, a preset condition is used to determine whether a first pixel point is an intermediate pixel point between a starting pixel point and an ending pixel point of a lane line. If the preset condition is met, the first pixel point is taken as an intermediate pixel point of the lane line and stored in a preset queue. If the preset condition is not met, the first pixel point is considered not to be an intermediate pixel point of the lane line, and is not stored in the preset queue, and the next first pixel point is determined.

[0059] In the embodiment of the present invention, based on the starting pixel point and the ending pixel point of the left lane line, and the first pixel point, if it is determined that the preset condition is met, the first pixel point is used as the middle pixel point of the left lane line. And based on the starting pixel point and the ending pixel point of the right lane line, and the first pixel point, if it is determined that the preset condition is met, the first pixel point is used as the middle pixel point of the right lane line.

[0060] In an embodiment of the present invention, the first pixel point is obtained by traversing the pixel points to be tested based on a preset order. The preset order can be an order from bottom to top and from left to right, or an order from bottom to top and from right to left, or an order from top to bottom and from left to right, or an order from top to bottom and from right to left.

[0061] Step S104: taking the middle pixel point as the next starting pixel point, and continuing to determine whether the preset condition is met, until the next first pixel point is the ending pixel point, all the found middle pixel points are drawn on the captured image.

[0062] In an embodiment of the present invention, when preset conditions are met, the first pixel point is used as the middle pixel point of the lane line, and the middle pixel point is used as the next starting pixel point, and combined with the next first pixel point and the ending pixel point, it is determined whether the next first pixel point is the middle pixel point of the lane line, and so on, until the next first pixel point traversed is the ending pixel point, then all the middle pixel points between the starting pixel point and the ending pixel point of the lane line are obtained, and then all the middle pixel points of the lane line as well as the starting pixel point and the ending pixel point are drawn on the captured image.

[0063] In an embodiment of the present invention, based on the binary image corresponding to the captured image, the starting pixel point and the ending pixel point of the lane line are found, and then combined with the preset conditions, when it is determined that the first pixel point traversed in the preset order is the middle pixel point between the starting pixel point and the ending pixel point, the first pixel point is used as the next starting pixel point, and combined with the next first pixel point, the steps of whether the preset conditions are met are continued to be executed until the next first pixel point is the ending pixel point, indicating that the search process for the middle pixel point of the lane line is completed, and finally all the pixel points of the lane line are drawn on the captured image to help the car owner correctly identify the lane line and ensure driving safety. In addition, this method of determining the lane line detection has low requirements on the hardware platform, thereby achieving the purpose of reducing a certain cost.

[0064] In some embodiments of the present invention, the preset conditions include:

[0065] The ground distance between the starting pixel point and the first pixel point is less than a distance threshold, and the ground distance between the first pixel point and the ending pixel point is less than the distance threshold, and a multiple of a first preset value of the ground distance between the starting pixel point and the ending pixel point is greater than the sum of the ground distance between the starting pixel point and the first pixel point and the ground distance between the first pixel point and the ending pixel point;

[0066] The distance threshold is determined based on the ground distance between the starting pixel point and the ending pixel point.

[0067] It should be noted that the ground distance between the starting pixel and the first pixel is less than the distance threshold, and the ground distance between the first pixel and the ending pixel is less than the distance threshold, in order to limit the first pixel to between the starting pixel and the ending pixel. In addition, the multiple of the first preset value of the ground distance between the starting pixel and the ending pixel is greater than the sum of the ground distance between the starting pixel and the first pixel and the ground distance between the first pixel and the ending pixel, in order to allow the lane line of the curve to be found. Based on this, when all three are satisfied, it means that the first pixel is an intermediate pixel between the starting pixel and the ending pixel.

[0068] In the embodiment of the present invention, the preset condition can be expressed as formula (1):

[0069] dist[i][k]<dist THRE And dist[k][j]<dist THRE And A*dist[i][j]>dist[i][k]+dist[k][j] (1)

[0070] Where i represents the starting pixel, k represents the first pixel, j represents the ending pixel, dist[i][k] represents the ground distance between the starting pixel and the first pixel, and dist THRE represents the distance threshold, dist[k][j] represents the ground distance between the first pixel and the end pixel, dist[i][j] represents the ground distance between the start pixel and the end pixel, and A is a first preset value. For example, the first preset value may be 1.3.

[0071] In the embodiment of the present invention, the distance threshold is determined based on the ground distance between the starting pixel and the ending pixel. When the preset conditions are met, the first pixel, i.e., the middle pixel, is used to update the starting pixel. Therefore, the distance threshold changes with the change of the starting pixel.

[0072] It should be noted that, for the left lane line, the preset conditions are the starting pixel point and the ending pixel point of the left lane line. For the right lane line, the preset conditions are the starting pixel point and the ending pixel point of the right lane line.

[0073] In some embodiments of the present invention, the preset condition further includes:

[0074] An absolute value of a difference between the first slope and the second slope is less than a slope threshold;

[0075] The first slope is determined based on the pixel coordinates of the starting pixel point and the pixel coordinates of the ending pixel point, and the second slope is determined based on the pixel coordinates of the starting pixel point and the pixel coordinates of the first pixel point.

[0076] It should be noted that, when the absolute value of the difference between the first slope and the second slope is less than the slope threshold, it indicates that the starting pixel point, the ending pixel point and the first pixel point are pixels on the same lane line.

[0077] In the embodiment of the present invention, the preset condition is also expressed as formula (2):

[0078] K L[R] -K i,k |<K THRE (2)

[0079] Among them, K L[R] represents the first slope, which is determined based on the pixel coordinates of the starting pixel point and the ending pixel point of the left lane line, or based on the pixel coordinates of the starting pixel point and the ending pixel point of the right lane line. i,k represents the second slope, which is determined based on the pixel coordinates of the starting pixel point of the left lane line and the pixel coordinates of the first pixel point, or based on the pixel coordinates of the starting pixel point of the right lane line and the pixel coordinates of the first pixel point. THRE Indicates the slope threshold.

[0080] In the implementation of the present invention, when both formula (1) and formula (2) are satisfied, the first pixel point is used as the middle pixel point of the lane line.

[0081] In some embodiments of the present invention, the lane line detection method further includes:

[0082] Based on the camera's internal and external parameter matrix, convert the second ground coordinate into a second pixel coordinate; wherein the abscissa of the second ground coordinate is zero, and the ordinate is not restricted; for example, the second ground coordinate is (0, h), where h is the lane line detection distance;

[0083] For the left lane line, the first pixel point is obtained by traversing the pixel points to be measured based on a preset order, and the horizontal coordinate of the first pixel point is less than or equal to the horizontal coordinate of the second pixel coordinate;

[0084] For the right lane line, the first pixel point is obtained by traversing the pixel points to be measured based on a preset order, and the horizontal coordinate of the first pixel point is greater than the horizontal coordinate of the second pixel coordinate.

[0085] Based on this, the number of pixels to be traversed can be reduced, thereby improving the speed of finding the middle pixel of the lane line.

[0086] In some embodiments of the present invention, the step of searching for a starting pixel point and an ending pixel point of a lane line from a plurality of pixels to be detected included in the binary image includes the following steps S201 to S203:

[0087] Step S201: based on the intrinsic and extrinsic parameter matrices of the camera, the pixel coordinates of a plurality of pixels to be measured are converted into ground coordinates.

[0088] In the embodiment of the present invention, the internal and external parameter matrices of the camera include an external parameter matrix and an internal parameter matrix. The external parameter matrix is ​​used to convert the world coordinate system (i.e., the vehicle coordinate system) and the camera coordinate system, and the internal parameter matrix is ​​used to convert the camera coordinate system and the pixel coordinate system. Among them, the coordinate origin of the pixel coordinate system is the upper left corner point. Based on this, the pixel coordinates of the pixel to be measured are converted into the corresponding camera coordinates using the internal parameter matrix, and then the camera matrix is ​​converted into the ground coordinates using the external parameter matrix.

[0089] Step S202: Based on the ground coordinates of the multiple pixels to be measured, determine the starting ground coordinates of the lane line on one side that is closest to the vehicle; and use the pixel point corresponding to the starting ground coordinate as the starting pixel point.

[0090] In the embodiment of the present invention, the lane marking on one side is a left lane marking or a right lane marking.

[0091] Step S203: Based on the starting ground coordinates of the lane line on one side and the lane width, determine the starting ground coordinates of the lane line on the other side.

[0092] In the embodiment of the present invention, based on the starting ground coordinates of the lane line on one side (left side or right side), the lane width units are moved right or left in the horizontal direction to obtain the starting ground coordinates of the lane line on the other side (right side or left side).

[0093] Step S204: Determine the end ground coordinates of each lane line based on the start ground coordinates of each lane line and the lane line detection distance; and use the pixel point corresponding to the end ground coordinate as the end pixel point.

[0094] In the embodiment of the present invention, based on the starting ground coordinates of the lane line on one side, the lane line detection distance is moved upward in the vertical direction by units to obtain the ending ground coordinates of the lane line on one side, and the pixel point corresponding to the ending ground coordinates of the lane line on one side is used as the ending pixel point. Based on the starting ground coordinates of the lane line on the other side, the lane line detection distance is moved upward in the vertical direction by units to obtain the ending ground coordinates of the lane line on the other side, and the pixel point corresponding to the ending ground coordinates of the lane line on the other side is used as the ending pixel point.

[0095] In some embodiments of the present invention, the step of determining the starting ground coordinates of the lane line on one side based on the ground coordinates of the plurality of pixels to be measured and a preset neighborhood radius includes the following steps S301 to S303:

[0096] Step S301: Selecting ground coordinates of a plurality of candidate pixel points located on one side of the vehicle from the ground coordinates of a plurality of candidate pixel points.

[0097] In the embodiment of the present invention, the ground coordinates are the coordinates of the ground points in the vehicle coordinate system. Therefore, the ground coordinates of the multiple pixels to be measured can be selected from the ground coordinates of the multiple pixels to be measured, and the ground coordinates of the multiple pixels to be measured can be selected from the left side of the vehicle, or the ground coordinates of the multiple pixels to be measured can be selected from the right side of the vehicle.

[0098] Step S302: Determine the distance between the ground coordinates of each candidate pixel and the ground coordinates of the vehicle.

[0099] In the embodiment of the present invention, the ground coordinates of the vehicle are (0, 0). Here, the square value of 0 is subtracted from the ordinate included in the ground coordinates of each pixel to be selected, and the square value of 0 is subtracted from the abscissa included in the ground coordinates of each pixel to be selected, and then the square root is taken after adding them to obtain the distance between the ground coordinates of the pixel to be selected and the ground coordinates of the vehicle.

[0100] Step S303: Based on the ground coordinates of the candidate pixel point corresponding to the minimum distance, determine the starting ground coordinates of the lane line on one side that is closest to the vehicle.

[0101] That is, the ground coordinates of the candidate pixel point corresponding to the minimum distance are the starting ground coordinates.

[0102] In some embodiments of the present invention, the preprocessing based on the captured image to obtain a binary image includes the following steps S401 to S403:

[0103] Step S401: based on the camera's internal and external parameter matrix and the lane line detection distance, the captured image is cropped to obtain a region of interest.

[0104] It should be noted that since the captured image includes the near ground area and the far area (including the ground and the sky), only the lane lines within the preset distance from the vehicle (i.e., the lane line detection distance) need to be detected. Therefore, based on the camera's internal and external parameter matrix and the lane line detection distance, the captured image is cropped to obtain the area of ​​interest, thereby reducing the amount of calculation and the calculation time.

[0105] Step S402: grayscale processing is performed on the region of interest to obtain a grayscale image.

[0106] In the embodiment of the present invention, grayscale processing refers to the process of converting a color image into a grayscale image. A grayscale image, also known as a grayscale image, is an image that contains only brightness information but not color information. In such an image, the brightness level of a pixel is usually quantized into a range of 0 to 255, where 0 represents black, 255 represents white, and the values ​​in between represent different levels of gray.

[0107] Specifically, if the captured image is an RGB image, the grayscale value of each pixel in the region of interest is calculated using formula (3), thereby obtaining a grayscale image including only brightness information.

[0108] Gray=0.299*R+0.578*G+0.114*B (3)

[0109] Step S403: binarize the grayscale image to obtain a binary image.

[0110] In the embodiment of the present invention, the binarization process, also known as the thresholding process, is mainly used to convert the grayscale image into an image containing only two pixel values, namely black and white (pixel values ​​255 and 0). The conversion process divides the pixels in the grayscale image into two categories based on one or more preset thresholds: pixels above the threshold are assigned one value (usually white 0), and pixels below the threshold are assigned another value (usually black 255).

[0111] In some embodiments of the present invention, the step of cropping the captured image based on the camera's internal and external parameter matrix and the lane line detection distance to obtain the region of interest includes the following steps S501 to S502:

[0112] Step S501: Determine a first pixel coordinate based on a first ground coordinate and an internal and external parameter matrix of the camera; wherein the horizontal coordinate of the first ground coordinate is zero, and the vertical coordinate is the lane line detection distance.

[0113] In the embodiment of the present invention, the internal and external parameter matrices of the camera include an external parameter matrix and an internal parameter matrix. The external parameter matrix is ​​used to convert from the world coordinate system (i.e., the vehicle coordinate system) to the camera coordinate system, and the internal parameter matrix is ​​used to convert from the camera coordinate system to the pixel coordinate system. The origin of the pixel coordinate system is the upper left corner point. Based on this, the first ground coordinate is converted into the corresponding camera coordinate using the external parameter matrix, and then the camera matrix is ​​converted into the first pixel coordinate using the internal parameter matrix.

[0114] In the embodiment of the present invention, the lane line detection distance is the lane line detection range.

[0115] Step S502: Using the ordinate of the first pixel coordinate as the cropping width, cropping the captured image from top to bottom to obtain a region of interest.

[0116] That is, the portion of the captured image corresponding to the cropping width is cropped from top to bottom to obtain the region of interest. Subsequent processing based on the region of interest can reduce the amount of calculation and thus reduce the calculation time.

[0117] Based on the above embodiments, the present invention provides a lane line detection device. Figure 6 Schematic diagram of the structure of the lane line detection device in an embodiment of the present invention Figure 1 ,like Figure 6 As shown, the lane line detection device includes: an image input module 60, an image preprocessing module 61, a lane line detection module 62 and a lane line output module 63;

[0118] Based on this, the present invention provides a lane line detection method based on the Floyd algorithm, specifically:

[0119] Image input module 60:

[0120] The main process is to load the captured images in RGB format into the memory. In the present invention, the Open Source Computer Vision Library (OpenCV) is used to read the multimedia files in the camera or hard disk, and the captured images in RGB format are read into the memory in frame order. After that, the program will perform image preprocessing on the captured images in RGB format.

[0121] Image preprocessing module 60:

[0122] First, crop the captured image in combination with the camera's internal and external parameter matrices to obtain the region of interest. When installing and calibrating the camera, extend forward from the center of the vehicle (usually the center of the rear axle), set up a coordinate system, and perform camera external parameter calibration. When selecting the region of interest (ROI), you can specify the height h of the detection range (i.e., the lane line detection distance) from bottom to top in combination with the image after the camera is fixed. Due to the principle of monocular camera ranging, the physical coordinate error of distant pixels calculated by the internal and external parameter matrices is large, so it is necessary to avoid selecting a detection range that is too large. This approach can reduce the amount of calculation and reduce the calculation time.

[0123] Secondly, cvtColor() is used to convert the image passing through the ROI area into a grayscale image. Conversion formula (3):

[0124] Gray=0.299*R+0.578*G+0.114*B (3)

[0125] Again, GaussianBlur() is used to reduce the noise interference of the grayscale image. Gaussian blur is very effective in smoothing impulse noise, and it can protect the sharp edges of the image. Weighted median filtering can improve the edge signal retention effect of median filtering. Since lane lines need to be detected based on the edge features of the image in the subsequent processing step (lane line detection), the invention uses Gaussian blur that can reduce noise interference and retain the original contour information of the image.

[0126] Finally, use Canny() to extract all edge information in the image. Since image noise can easily affect the Canny algorithm, Canny() also includes a denoising process. Since the invention uses the edge features of the image to identify lane lines, Canny is crucial for subsequent processing (lane line recognition). After processing, a binary image with pixel values ​​of 0 or 255 is obtained (such as Figure 7 shown).

[0127] Lane detection module 61:

[0128] Combined with the camera's internal and external parameter matrices, the pixel coordinates of multiple pixels to be measured (pixels with a pixel value of 255) in the binary image are converted into ground coordinates.

[0129] From the ground coordinates of multiple pixels to be measured, determine the starting ground coordinates of the left lane line closest to the vehicle. Here, the left lane line is taken as an example. The pixel point corresponding to the starting ground coordinates of the left lane line is the starting pixel point of the left lane line.

[0130] Based on the starting ground coordinates of the left lane line, move lane width units to the right in the horizontal direction to obtain the starting ground coordinates of the right lane line.

[0131] That is, the starting ground coordinates of the left lane line and the right lane line are obtained.

[0132] Further, based on the starting ground coordinate of the left lane line, the lane line detection distance is moved upward in the vertical direction by units to obtain the ending ground coordinate of the left lane line, and the pixel point corresponding to the ending ground coordinate of the left lane line is used as the ending pixel point. Based on the starting ground coordinate of the right lane line, the lane line detection distance is moved upward in the vertical direction by units to obtain the ending ground coordinate of the right lane line, and the pixel point corresponding to the ending ground coordinate of the right lane line is used as the ending pixel point.

[0133] Next, determine whether the ground coordinates of the starting pixel point and the ending pixel point of the left or right lane line, as well as the ground coordinates of the first pixel point, satisfy formula (4):

[0134] dist[i][k]<dist THRE And dist[k][j]<dist THRE

[0135] And 1.3*dist[i][j]>dist[i][k]+dist[k][j](4)

[0136] Where i represents the starting pixel, k represents the first pixel, j represents the ending pixel, dist[i][k] represents the ground distance between the starting pixel and the first pixel, and dist THRE represents the distance threshold, dist[k][j] represents the ground distance between the first pixel and the end pixel, and dist[i][j] represents the ground distance between the start pixel and the end pixel.

[0137] The first pixel point is obtained by traversing the pixel points to be tested sequentially from bottom to top and from left to right.

[0138] On the basis of satisfying formula (1), further determine whether formula (2) is satisfied:

[0139] K L[R] -K i,k |<K THRE (2)

[0140] Among them, K L[R] represents the first slope, which is determined based on the pixel coordinates of the starting pixel point and the ending pixel point of the left or right lane line. i,k represents the second slope, which is determined based on the pixel coordinates of the starting pixel point of the left or right lane line and the pixel coordinates of the first pixel point. THRE Indicates the slope threshold.

[0141] If both formula (1) and formula (2) are satisfied, the first pixel point is stored in the preset queue as the middle pixel point of the lane line, and the middle pixel point is used as the starting pixel point for the next time, and whether formulas (1) and (2) are satisfied is continued until the next first pixel point is the ending pixel point. Finally, the lane line output module 63 draws all the middle pixel points in the preset queue on the captured image (such as Figure 8 shown).

[0142] It should be noted that the predetermined lane line is a straight line parallel to the vehicle body. In actual detection, the vehicle and the lane line may not be parallel, or the current lane may be a curve, so the detection point is not on the predetermined lane line.

[0143] The image input module 60 , the image preprocessing module 61 , and the lane line detection module 62 correspond to the processing unit, and the lane line output module 63 corresponds to the drawing unit.

[0144] An embodiment of the present invention provides a lane line detection device. Fig. 9 Schematic diagram of the structure of the lane line detection device in an embodiment of the present invention Figure 2 ,like Fig. 9 As shown, the lane line detection device 90 includes:

[0145] The processing unit 901 is used to perform preprocessing based on the captured image to obtain a binary image;

[0146] The processing unit 901 is further configured to find a starting pixel point and an ending pixel point of a lane line from a plurality of pixels to be tested included in the binary image;

[0147] The processing unit 901 is further configured to determine, based on the starting pixel point and the ending pixel point of the lane line, and the first pixel point, that the first pixel point is used as the middle pixel point of the lane line when a preset condition is met; wherein the first pixel point is obtained by traversing the pixel points to be tested in a preset order;

[0148] The processing unit 901 is also used to use the intermediate pixel point as the next starting pixel point, and continue to determine whether the preset condition is met until the next first pixel point is the ending pixel point. The drawing unit 902 is used to draw all the found intermediate pixel points on the captured image.

[0149] In an embodiment of the present invention, based on the binary image corresponding to the captured image, the starting pixel point and the ending pixel point of the lane line are found, and then combined with the preset conditions, when it is determined that the first pixel point traversed in the preset order is the middle pixel point between the starting pixel point and the ending pixel point, the first pixel point is used as the next starting pixel point, and combined with the next first pixel point, the steps of whether the preset conditions are met are continued to be executed until the next first pixel point is the ending pixel point, indicating that the search process for the middle pixel point of the lane line is completed, and finally all the pixel points of the lane line are drawn on the captured image to help the car owner correctly identify the lane line and ensure driving safety. In addition, this method of determining the lane line detection has low requirements on the hardware platform, thereby achieving the purpose of reducing a certain cost.

[0150] In some embodiments of the present invention, the preset conditions include:

[0151] The ground distance between the starting pixel point and the first pixel point is less than a distance threshold, and the ground distance between the first pixel point and the ending pixel point is less than the distance threshold, and a multiple of a first preset value of the ground distance between the starting pixel point and the ending pixel point is greater than the sum of the ground distance between the starting pixel point and the first pixel point and the ground distance between the first pixel point and the ending pixel point;

[0152] The distance threshold is determined based on the ground distance between the starting pixel point and the ending pixel point.

[0153] In some embodiments of the present invention, the preset condition further includes:

[0154] An absolute value of a difference between the first slope and the second slope is less than a slope threshold;

[0155] The first slope is determined based on the pixel coordinates of the starting pixel point and the pixel coordinates of the ending pixel point, and the second slope is determined based on the pixel coordinates of the starting pixel point and the pixel coordinates of the first pixel point.

[0156] In some embodiments of the present invention, the processing unit 901 is further configured to convert the pixel coordinates of the plurality of pixels to be measured into ground coordinates based on an internal and external parameter matrix of the camera;

[0157] Based on the ground coordinates of the plurality of pixels to be measured, determine the starting ground coordinate of the lane line on one side that is closest to the vehicle; and use the pixel point corresponding to the starting ground coordinate as the starting pixel point;

[0158] Based on the starting ground coordinates of the lane line on one side and the lane width, determine the starting ground coordinates of the lane line on the other side;

[0159] Based on the starting ground coordinates of each lane line and the lane line detection distance, the ending ground coordinates of each lane line are determined; and the pixel point corresponding to the ending ground coordinate is used as the ending pixel point.

[0160] In some embodiments of the present invention, the processing unit 901 is further configured to select ground coordinates of a plurality of candidate pixel points located on one side of the vehicle from the ground coordinates of the plurality of candidate pixel points;

[0161] Determine the distance between the ground coordinates of each candidate pixel and the ground coordinates of the vehicle;

[0162] Based on the ground coordinates of the candidate pixel points corresponding to the minimum distance, the starting ground coordinates of the lane line on one side closest to the vehicle are determined.

[0163] In some embodiments of the present invention, the processing unit 901 is further configured to crop the captured image based on the camera's internal and external parameter matrix and the lane line detection distance to obtain a region of interest;

[0164] Performing grayscale processing on the region of interest to obtain a grayscale image;

[0165] The grayscale image is binarized to obtain the binary image.

[0166] In some embodiments of the present invention, the processing unit 901 is further configured to determine a first pixel coordinate based on a first ground coordinate and an internal and external parameter matrix of the camera; wherein the abscissa of the first ground coordinate is zero, and the ordinate is the lane line detection distance;

[0167] The vertical coordinate of the first pixel coordinate is used as the cropping width, and the captured image is cropped from top to bottom to obtain a region of interest.

[0168] The embodiment of the present invention further provides another electronic device, Fig.10 FIG. 1 is a schematic diagram of the structure of an electronic device according to an embodiment of the present invention. Fig.10 As shown, the electronic device 100 includes: a processor 1001 and a memory 1002 configured to store a computer program that can be run on the processor;

[0169] The processor 1001 is configured to execute the method steps in the aforementioned embodiment when running a computer program.

[0170] Of course, in practical applications, Fig.10 As shown, the components in the electronic device 100 are coupled together via a bus system 1003. It is understood that the bus system 1003 is used to realize the connection and communication between these components. In addition to the data bus, the bus system 1003 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, Fig.10 Various buses are labeled as bus system 1003.

[0171] In practical applications, the processor may be at least one of an application-specific integrated circuit (ASIC), a digital signal processing device (DSPD), a programmable logic device (PLD), a field-programmable gate array (FPGA), a controller, a microcontroller, and a microprocessor. It is understandable that for different devices, the electronic device used to implement the functions of the processor may also be other, and the embodiments of the present invention do not specifically limit this.

[0172] The above-mentioned memory can be a volatile memory (volatile memory), such as a random access memory (RAM); or a non-volatile memory (non-volatile memory), such as a read-only memory (ROM), a flash memory, a hard disk (HDD) or a solid-state drive (SSD); or a combination of the above-mentioned types of memory, and provide instructions and data to the processor.

[0173] In an exemplary embodiment, an embodiment of the present invention further provides a computer-readable storage medium for storing a computer program.

[0174] Optionally, the computer-readable storage medium can be applied to any one of the methods in the embodiments of the present invention, and the computer program enables the computer to execute the corresponding processes implemented by the processor in each method in the embodiments of the present invention. For the sake of brevity, they are not described here.

[0175] In the several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0176] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0177] In addition, all functional units in the embodiments of the present invention may be integrated into one processing module, or each unit may be a separate unit, or two or more units may be integrated into one unit; the above integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional units. A person of ordinary skill in the art may understand that all or part of the steps of implementing the above method embodiments may be completed by hardware related to program instructions, and the above program may be stored in a computer-readable storage medium, which, when executed, executes the steps of the above method embodiments; and the above storage medium includes various media that can store program codes, such as mobile storage devices, read-only memories (ROM), random access memories (RAM), magnetic disks or optical disks.

[0178] The methods disclosed in the several method embodiments provided by the present invention can be arbitrarily combined without conflict to obtain new method embodiments.

[0179] The features disclosed in several product embodiments provided by the present invention can be arbitrarily combined without conflict to obtain new product embodiments.

[0180] The features disclosed in several method or device embodiments provided by the present invention can be arbitrarily combined without conflict to obtain new method embodiments or device embodiments.

[0181] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A lane line detection method, characterized in that: The lane line detection method comprises: Preprocessing is performed based on the captured image to obtain a binary image; Finding the starting pixel point and the ending pixel point of the lane line from the multiple pixels to be tested included in the binary image; Based on the starting pixel point and the ending pixel point of the lane line, and the first pixel point, if a preset condition is met, the first pixel point is used as the middle pixel point of the lane line; wherein the first pixel point is obtained by traversing the pixel points to be tested in a preset order; The intermediate pixel point is used as the next starting pixel point, and whether the preset condition is satisfied is continuously determined until the next first pixel point is the ending pixel point, and all the intermediate pixel points found are then drawn on the captured image.

2. The lane line detection method according to claim 1, characterized in that: The preset conditions include: The ground distance between the starting pixel point and the first pixel point is less than a distance threshold, and the ground distance between the first pixel point and the ending pixel point is less than the distance threshold, and a multiple of a first preset value of the ground distance between the starting pixel point and the ending pixel point is greater than the sum of the ground distance between the starting pixel point and the first pixel point and the ground distance between the first pixel point and the ending pixel point; The distance threshold is determined based on the ground distance between the starting pixel point and the ending pixel point.

3. The lane line detection method according to claim 2, characterized in that: The preset conditions also include: An absolute value of a difference between the first slope and the second slope is less than a slope threshold; The first slope is determined based on the pixel coordinates of the starting pixel point and the pixel coordinates of the ending pixel point, and the second slope is determined based on the pixel coordinates of the starting pixel point and the pixel coordinates of the first pixel point.

4. The lane line detection method according to any one of claims 1 to 3, characterized in that: The step of searching for a starting pixel point and an ending pixel point of a lane line from a plurality of pixels to be tested included in the binary image comprises: Based on the internal and external parameter matrix of the camera, the pixel coordinates of the plurality of pixels to be measured are converted into ground coordinates; Based on the ground coordinates of the plurality of pixels to be measured, determine the starting ground coordinate of the lane line on one side that is closest to the vehicle; and use the pixel point corresponding to the starting ground coordinate as the starting pixel point; Based on the starting ground coordinates of the lane line on one side and the lane width, determine the starting ground coordinates of the lane line on the other side; Based on the starting ground coordinates of each lane line and the lane line detection distance, the ending ground coordinates of each lane line are determined; and the pixel point corresponding to the ending ground coordinate is used as the ending pixel point.

5. The lane line detection method according to claim 4, characterized in that: The step of determining the starting ground coordinates of the lane line on one side closest to the vehicle based on the ground coordinates of the plurality of pixels to be measured comprises: Selecting ground coordinates of a plurality of candidate pixel points located on one side of the vehicle from the ground coordinates of the plurality of candidate pixel points; Determine the distance between the ground coordinates of each candidate pixel and the ground coordinates of the vehicle; Based on the ground coordinates of the candidate pixel points corresponding to the minimum distance, the starting ground coordinates of the lane line on one side closest to the vehicle are determined.

6. The lane line detection method according to any one of claims 1 to 3, characterized in that: The preprocessing based on the captured image to obtain a binary image includes: Based on the camera's internal and external parameter matrix and the lane line detection distance, the captured image is cropped to obtain a region of interest; Performing grayscale processing on the region of interest to obtain a grayscale image; The grayscale image is binarized to obtain the binary image.

7. The lane line detection method according to claim 6, characterized in that: The captured image is cropped based on the camera's internal and external parameter matrix and the lane line detection distance to obtain an area of ​​interest, including: Determine a first pixel coordinate based on a first ground coordinate and an internal and external parameter matrix of the camera; wherein the abscissa of the first ground coordinate is zero and the ordinate is the lane line detection distance; The vertical coordinate of the first pixel coordinate is used as the cropping width, and the captured image is cropped from top to bottom to obtain a region of interest.

8. A lane line detection device, characterized in that: The lane line detection device comprises: A processing unit, used for performing preprocessing based on the captured image to obtain a binary image; The processing unit is further used to find the starting pixel point and the ending pixel point of the lane line from the multiple pixels to be tested included in the binary image; The processing unit is further configured to determine, based on the starting pixel point and the ending pixel point of the lane line, and the first pixel point, that the first pixel point is used as the middle pixel point of the lane line when a preset condition is met; wherein the first pixel point is obtained by traversing the pixel points to be tested in a preset order; The processing unit is also used to use the intermediate pixel point as the next starting pixel point, and continue to determine whether the preset condition is met until the next first pixel point is the ending pixel point. The drawing unit is used to draw all the found intermediate pixel points on the captured image.

9. An electronic device, characterized in that: The electronic device comprises: a processor and a memory configured to store a computer program capable of running on the processor, Wherein, the processor is configured to execute the steps of the lane line detection method according to any one of claims 1 to 7 when running the computer program.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the lane line detection method described in any one of claims 1 to 7 are implemented.

Images