Lane line detection method, device, equipment and storage medium

Abstract

The application discloses a lane line detection method and device, equipment and a storage medium, and relates to the technical field of image processing and deep learning. The lane line detection method comprises the following steps: acquiring a target detection image, wherein at least one lane line is included in the target detection image, and the lane line has a corresponding type label; performing feature extraction on the target detection image to obtain lane line binary images corresponding to lane lines of various types in the target detection image; performing connected domain analysis on the lane line binary images based on the type label, disconnecting connected domains of lane line binary images that meet a preset condition at intersection positions, and the preset condition being that the shape of the connected domains of the lane line binary images meets a preset target shape; sampling feature points at a preset interval based on the lane line binary images after the connected domains are disconnected at the intersection positions to obtain lane line feature points corresponding to lane lines of various types; and generating lane line information corresponding to lane lines of various types based on the lane line feature points corresponding to the lane lines of various types.

Description

Technical Field

[0001] This application belongs to the field of image processing and deep learning technology, and in particular relates to a lane line detection method, apparatus, device and storage medium. Background Technology

[0002] Lane detection is one of the key technologies in autonomous driving and intelligent transportation systems, enabling vehicle navigation and control systems to better understand and control vehicle movement.

[0003] Most existing lane detection methods use convolutional neural networks or recurrent neural networks for learning and detection.

[0004] However, this method has poor lane detection accuracy and low accuracy results under conditions of complex lanes and multiple lane markings. Summary of the Invention

[0005] This application provides a lane line detection method, apparatus, device, and storage medium, which can improve the detection accuracy of lane lines in complex lanes and various types of lane lines.

[0006] One aspect of this application provides a lane line detection method, the method comprising:

[0007] Acquire a target detection image, which includes at least one lane line and the lane line has a corresponding type label;

[0008] Feature extraction is performed on the target detection image to obtain the binary map of lane lines corresponding to each type of lane line in the target detection image;

[0009] Based on type labels, a connectivity analysis is performed on the binary graphs of each lane line. The connectivity of the lane line binary graphs that meet the preset conditions is broken at the intersection points. The preset condition is that the shape of the connectivity of the lane line binary graphs conforms to the preset target shape.

[0010] Based on the binary images of each lane after the connected components are broken at the intersection, feature points are sampled at preset intervals to obtain lane feature points corresponding to each type of lane.

[0011] Based on the lane feature points corresponding to each type of lane line, lane line information corresponding to each type of lane line is generated.

[0012] One aspect of this application provides a lane line detection device, the device comprising:

[0013] The image acquisition module is used to acquire a target detection image, which includes at least one lane line and the lane line has a corresponding type label.

[0014] The feature extraction module is used to extract features from the target detection image to obtain the binary map of lane lines corresponding to each type of lane line in the target detection image;

[0015] The connected component analysis module is used to perform connected component analysis on the binary maps of each lane line based on the type label. It breaks the connected components of the binary maps of lane lines that meet the preset conditions at the intersection points. The preset condition is that the shape of the connected components of the binary maps of lane lines conforms to the preset target shape.

[0016] The feature point sampling module is used to sample feature points at preset intervals based on the binary images of each lane line after the connected components are broken at the intersection, so as to obtain the lane line feature points corresponding to each type of lane line.

[0017] The information generation module is used to generate lane line information for each type of lane line based on the lane line feature points corresponding to each type of lane line.

[0018] In one aspect of this application, an electronic device is provided, the device comprising: a memory and a program or instructions stored in the memory and executable on a processor, wherein when the program or instructions are executed by the processor, they implement the lane line detection method provided in any aspect of the above-described embodiments of this application.

[0019] In one aspect of the embodiments of this application, a readable storage medium is provided, on which a program or instructions are stored, which, when executed by a processor, implement the lane line detection method provided in any aspect of the embodiments of this application described above.

[0020] In one aspect of the embodiments of this application, a computer program product is provided, wherein the instructions in the computer program product, when executed by the processor of an electronic device, cause the electronic device to perform the lane line detection method provided in any aspect of the embodiments of this application described above.

[0021] The lane detection method provided in this application involves acquiring a target detection image, which includes at least one lane line, each lane line having a corresponding type label. Then, feature extraction is performed on the target detection image to obtain binary maps of lane lines corresponding to each type of lane line. Next, connected component analysis is performed on each lane line binary map based on the type label. If the shape of the connected component of the lane line binary map conforms to a preset target shape, the connected component of the corresponding lane line binary map is broken at the intersection point. Then, based on each lane line binary map after the connected component is broken at the intersection point, feature points are sampled at preset intervals to obtain lane line feature points corresponding to each type of lane line. Finally, based on the lane line feature points corresponding to each type of lane line, lane line information corresponding to each type of lane line is generated. Thus, this application embodiment, by performing connected component analysis on each lane line binary map, breaks the connected component of the corresponding lane line binary map at the intersection point if the shape of the connected component conforms to a preset target shape. It effectively avoids the problem of lane lines being incorrectly fitted into a single straight line due to the intersection of lane lines in complex lanes and various types of lane lines, thereby improving the detection accuracy of lane lines in complex lanes and various types of lane lines. Attached Figure Description

[0022] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 This is a flowchart illustrating an embodiment of a lane line detection method provided in this application;

[0024] Figure 2 This is a flowchart illustrating another embodiment of the lane line detection method provided in this application;

[0025] Figure 3 This is a flowchart illustrating another embodiment of the lane line detection method provided in this application;

[0026] Figure 4 This is a schematic diagram of the structure of an embodiment of the lane line detection device provided in this application;

[0027] Figure 5 This is a schematic diagram of the structure of an embodiment of the lane line detection device provided in this application. Detailed Implementation

[0028] The features and exemplary embodiments of various aspects of this application will be described in detail below. To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain this application and not to limit it. For those skilled in the art, this application can be implemented without some of these specific details. The following description of the embodiments is merely to provide a better understanding of this application by illustrating examples.

[0029] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes said element.

[0030] The acquisition, storage, use, and processing of data in this application all comply with the relevant provisions of national laws and regulations.

[0031] In related technologies, most existing lane detection methods use convolutional neural networks or recurrent neural networks for learning and detection. However, this method suffers from poor lane detection accuracy and low accuracy of detection results under conditions of complex lanes and multiple lane line types.

[0032] The purpose of this application is to provide a lane line detection method, apparatus, device, and storage medium. The lane line detection method provided in this application involves acquiring a target detection image, wherein the target detection image includes at least one lane line, and each lane line has a corresponding type label. Then, feature extraction is performed on the target detection image to obtain lane line binary maps corresponding to each type of lane line in the target detection image. Next, connected component analysis is performed on each lane line binary map based on the type label. If the shape of the connected component of the lane line binary map conforms to a preset target shape, the connected component of the corresponding lane line binary map is broken at the intersection point. Then, based on each lane line binary map after the connected component is broken at the intersection point, feature points are sampled at preset intervals to obtain lane line feature points corresponding to each type of lane line. Finally, based on the lane line feature points corresponding to each type of lane line, lane line information corresponding to each type of lane line is generated. Thus, this application embodiment, by performing connected component analysis on each lane line binary map, breaks the connected component of the corresponding lane line binary map at the intersection point if the shape of the connected component of the lane line binary map conforms to a preset target shape. It effectively avoids the problem of incorrectly fitting a single straight line due to the intersection of lane lines in complex lanes and various types of lane lines, thereby improving the detection accuracy of lane lines in complex lanes and various types of lane lines.

[0033] The following describes specific embodiments of the lane line detection method, apparatus, device, and storage medium provided in this application. The lane line detection method will be described first.

[0034] Figure 1 A flowchart of a lane line detection method is provided. This lane line detection method is applied to a server and may include the following steps S101 to S105.

[0035] S101, acquire a target detection image, the target detection image includes at least one lane line, and the lane line has a corresponding type label.

[0036] In this embodiment, the target detection image is a detection image containing lane lines. The target detection image includes at least one lane line.

[0037] Each lane marking has a corresponding type label, which indicates the type of the lane marking. For example, the type label may include single solid line type, single dashed line type, double solid line type, double dashed line type, and stop line type.

[0038] For example, the server acquires target detection images around the target vehicle through the camera, and assigns corresponding type labels to each lane line in each target detection image according to the type to which each lane line belongs.

[0039] S102, perform feature extraction on the target detection image to obtain the binary map of lane lines corresponding to each type of lane line in the target detection image.

[0040] In this embodiment, the lane line binary image is a binary image generated based on the lane lines in the target detection image.

[0041] For example, the server extracts lane line features from the acquired target detection image using a semantic segmentation model, thereby obtaining binary lane line maps corresponding to each type of lane line in the target detection image.

[0042] S103, perform connectivity analysis on each lane line binary graph based on type labels, and disconnect the connectivity of lane line binary graphs that meet preset conditions at the intersection points. The preset condition is that the shape of the connectivity of the lane line binary graph conforms to the preset target shape.

[0043] In this embodiment, the preset condition is that the shape of the connected region of the lane line binary graph conforms to the preset target shape.

[0044] The preset target shape is one that is likely to cause large errors in linear fitting. For example, the preset target shape may include a Y-shape and a V-shape.

[0045] For example, the server performs connectivity analysis on the binary map of lane lines based on the type labels corresponding to the lane lines, thereby determining the shape of the connected components corresponding to the binary map of lane lines. If the shape of the connected components corresponding to the binary map of lane lines is Y-shaped or V-shaped, the connected components of the binary map of lane lines are broken at the intersection points so that there are no intersecting areas in the shape of the connected components corresponding to the binary map of lane lines.

[0046] S104. Based on the binary images of each lane line after the connected domain is broken at the intersection, feature points are sampled at preset intervals to obtain the lane line feature points corresponding to each type of lane line.

[0047] In this embodiment, the preset interval is a pre-defined sampling interval for lane line feature points. For example, the preset interval can be set to sample 1 point every 5 pixels.

[0048] For example, the server samples feature points at preset intervals based on the binary map of lane lines after the connected components are broken at the intersection, and determines each sampled feature point as the lane line feature point of the corresponding lane line.

[0049] S105, Based on the lane line feature points corresponding to each type of lane line, generate lane line information corresponding to each type of lane line.

[0050] In this embodiment, lane line information refers to information characterizing the basic conditions of lane lines. For example, lane line information may include the start and end points of lane lines, the lane line fitting equation, and the cluster point categories of lane lines.

[0051] For example, the server determines the start and end points of the lane line, the lane line fitting equation, and the cluster point category of the lane line based on the feature points of each lane line, thereby generating the lane line information corresponding to the lane line.

[0052] This embodiment acquires a target detection image, which includes at least one lane line, and each lane line has a corresponding type label. Then, feature extraction is performed on the target detection image to obtain binary maps of lane lines corresponding to each type of lane line. Next, connected component analysis is performed on each lane line binary map based on the type label. If the shape of the connected component of the lane line binary map conforms to a preset target shape, the connected component of the corresponding lane line binary map is broken at the intersection point. Then, based on each lane line binary map after the connected component is broken at the intersection point, feature points are sampled at preset intervals to obtain lane line feature points corresponding to each type of lane line. Finally, based on the lane line feature points corresponding to each type of lane line, lane line information corresponding to each type of lane line is generated. Thus, this embodiment of the application performs connected component analysis on each lane line binary map, and if the shape of the connected component of the lane line binary map conforms to a preset target shape, the connected component of the corresponding lane line binary map is broken at the intersection point. It effectively avoids the problem of incorrectly fitting a single straight line due to the intersection of lane lines in complex lanes and various types of lane lines, thereby improving the detection accuracy of lane lines in complex lanes and various types of lane lines.

[0053] As an optional embodiment, S102 may specifically include:

[0054] Feature extraction is performed on the target detection image using a semantic segmentation model to generate a lane line mask image;

[0055] Based on the type labels, the masks corresponding to each type of lane line in the lane line mask image are extracted to generate a binary image of each type of lane line.

[0056] In this embodiment, the server inputs the target detection image into the trained semantic segmentation model to obtain the semantic segmentation result corresponding to the target detection image, and generates a lane line mask map of the same size as the target detection image based on the semantic segmentation result.

[0057] The type label of the lane line corresponding to each pixel is obtained based on the lane line mask image. Then, the mask corresponding to each type label is extracted to generate a binary image of the lane line with the corresponding type label.

[0058] As an example, a semantic segmentation model can be trained using the following methods:

[0059] Multiple target training images are acquired, each containing at least one lane line, and each lane line has a corresponding type label. The target training images are then labeled according to the type labels: background pixels are labeled 0, single solid line pixels are labeled 1, single dashed line pixels are labeled 2, double solid line pixels are labeled 3, double dashed line pixels are labeled 4, and stop line pixels are labeled 5. These labeled target training images are then divided into a test set and a training set. The semantic segmentation model is trained using these target training images from both the test and training sets.

[0060] In this embodiment, a lane line mask map is generated using a semantic segmentation model, and a corresponding lane line binary map is generated based on the lane line mask map. This helps to determine the shape of the corresponding connected components based on the lane line binary map, thereby improving the detection accuracy of lane lines in complex lanes and various lane line types.

[0061] As an optional embodiment, S103 may specifically include:

[0062] Based on type labels, perform connectivity analysis on the binary graphs of each lane line to determine whether the shape of the connected component corresponding to each lane line binary graph conforms to the preset target shape;

[0063] If the shape of the connected component corresponding to the lane line binary image conforms to the preset target shape, then based on the intersection position of the connected component, each pixel in the lane line binary image corresponding to the intersection position of the connected component is assigned zero.

[0064] In this embodiment, the server performs connected component analysis on the binary graph of lane lines based on the type label corresponding to the lane lines, and determines whether the shape of the connected component corresponding to the binary graph of lane lines is Y-shaped or V-shaped.

[0065] When the shape of the connected region corresponding to the lane line binary graph is Y-shaped or V-shaped, the intersection point position is determined according to the shape of the connected region corresponding to the lane line binary graph.

[0066] Then, based on the intersection point location, the corresponding intersection point and all pixels within the preset target distance of the intersection point are set to zero in the corresponding lane line binary map, thereby breaking the connected components of the lane line binary map at the intersection point location.

[0067] In this embodiment, when the shape of the connected component corresponding to the lane line binary image conforms to the preset target shape, each pixel point corresponding to the intersection point of the lane line binary image with the connected component is assigned zero, which helps to disconnect the connected component of the lane line binary image at the intersection point, thereby improving the detection accuracy of lane lines in complex lanes and various types of lane lines.

[0068] As an optional embodiment, connectivity analysis is performed on the binary graphs of each lane line based on type labels to determine whether the shape of the connected component corresponding to each lane line binary graph conforms to a preset target shape. Specifically, this may include:

[0069] Perform connectivity analysis on the lane line binary graph to obtain the coordinate range information of the first connected component and the connected component type label information corresponding to the lane line binary graph;

[0070] The direction of the first lane line is determined based on the coordinate range information of the first connected domain;

[0071] Scan the connected components along the direction of the first lane line to determine the number of times the connected component type label information changes;

[0072] Based on the number of times the connected component type label information changes, determine whether the shape of the connected component corresponding to the lane line binary map conforms to the preset target shape.

[0073] In this embodiment, the first connected domain coordinate range information is used to characterize the coordinate range corresponding to the connected domain.

[0074] The type label information of the connected component is associated with the type label of the lane line, and is used to characterize the type label of the lane line corresponding to the connected component.

[0075] As an example, such as Figure 2 As shown, the server performs connectivity analysis on the lane line binary graph to obtain the labels, coordinate ranges, and area information of the connected components corresponding to the lane line binary graph. Then, the lane line direction is determined based on the coordinate ranges of the connected components.

[0076] When the lane line direction is biased towards the x-axis, the connected region is scanned along the x-axis and sampled according to a preset scan interval; when the lane line direction is biased towards the y-axis, the connected region is scanned along the y-axis and sampled according to a preset scan interval.

[0077] Based on the sampling points obtained, determine whether there are any changes in the connected component labels within the current scanning axis. If the number of times the connected component labels change reaches three, it means that there are at least two points belonging to this connected component on the scanning axis that are disconnected in the middle, and the connected component status is recorded as "disconnected". If the number of times the connected component labels change is less than three, it means that there are no two points belonging to this connected component that are disconnected in the middle on the scanning axis, and the connected component status is recorded as "not disconnected".

[0078] If the state of a connected component changes between "disconnected" and "not disconnected" during the scan, it proves that the connected component is Y-shaped or V-shaped, and the location of the change is the location of the intersection.

[0079] If no change in the state of the connected component between "disconnected" and "not disconnected" occurs during the scan, it proves that the connected component is not Y-shaped or V-shaped.

[0080] In this embodiment, based on the number of times the connected component type label information changes, it is possible to determine whether the shape of the connected component corresponding to the lane line binary graph conforms to the preset target shape, thereby helping to improve the detection accuracy of lane lines in complex lanes and various types of lane lines.

[0081] As an optional embodiment, S104 may specifically include:

[0082] Perform connectivity analysis on the binary graphs of each lane line after the connectivity is broken at the intersection point to obtain the coordinate range information of the second connected domain corresponding to the binary graphs of each lane line after the intersection point is broken.

[0083] Based on the coordinate range information of the second connected domain, the direction of the second lane line is determined;

[0084] Scan the connected domain along the direction of the second lane line, and sample feature points at preset intervals to obtain lane line feature points corresponding to each type of lane line.

[0085] In this embodiment, as an example, such as Figure 3 As shown, the server performs connectivity analysis on the lane line binary graph to obtain the labels, coordinate ranges, and area information of the connected components corresponding to the lane line binary graph. Then, the lane line direction is determined based on the coordinate ranges of the connected components.

[0086] When the lane line direction is biased towards the x-axis, the connected region is scanned along the x-axis and sampled at a preset sampling interval; when the lane line direction is biased towards the y-axis, the connected region is scanned along the y-axis and sampled at a preset sampling interval. By sampling at the preset sampling interval, the sampled feature points are obtained.

[0087] Specifically, when scanning the connected region along the x-axis and sampling at a preset sampling interval, the y-coordinates of each point at the same x-coordinate are obtained, the average value of the y-coordinates of each point at the same x-coordinate is calculated, and the corresponding sampled feature points are obtained based on the average value.

[0088] When scanning the connected region along the y-axis and sampling at a preset sampling interval, the x-coordinates of points at the same y-coordinate are obtained. The average value of the x-coordinates of points at the same y-coordinate is calculated, and the corresponding sampled feature points are obtained based on the average value. Due to the characteristic of near objects appearing larger and far objects appearing smaller, it can be preset that the y-axis sampling of nearby points is sparser and the y-axis sampling of distant points is denser.

[0089] In this embodiment, based on the binary images of each lane line after the intersection point is broken, lane line feature points corresponding to each type of lane line are sampled, which helps to generate lane line information corresponding to each type of lane line based on the lane line feature points.

[0090] As an optional embodiment, after S104, the lane line detection method may further include:

[0091] Based on the intrinsic and extrinsic parameter information of the image acquisition device, the lane feature points corresponding to each type of lane line are transformed into lane position points in the corresponding world coordinate system.

[0092] Cluster the lane line location points in the world coordinate system corresponding to each type of lane line based on density to generate lane line instances of each type of lane line in the world coordinate system.

[0093] S105 specifically includes:

[0094] Based on lane line instances of each type in the world coordinate system, generate lane line information corresponding to each type of lane line.

[0095] In this embodiment, the internal and external parameter information of the image acquisition device includes the internal parameter information and the external parameter information of the image acquisition device.

[0096] The intrinsic parameter information of an image acquisition device refers to parameters related to the device's own characteristics. For example, the intrinsic parameter information of an image acquisition device may include the focal length and the number of pixels.

[0097] The extrinsic parameters of an image acquisition device are parameters related to the world coordinate system. For example, the extrinsic parameters of an image acquisition device may include the position of the image acquisition device and the direction of its rotation.

[0098] A lane line instance is a set of lane line location points that represent lane lines, including multiple lane line location points corresponding to the same type of lane line.

[0099] For example, the server first converts the acquired lane line feature points into lane line position points in the corresponding world coordinate system based on the internal and external parameter information of the image acquisition device.

[0100] Then, the lane line location points of the same type are sorted according to their coordinates. The lane line location points of each type are clustered according to their coordinates. All points in the feature point set are traversed. A lane line instance is created from the first point. Each traversed point is checked to see if it matches the last point of any existing lane line instance.

[0101] The matching condition is whether the distance in the world coordinate system is within the matching threshold range. If the distance is within the matching threshold range, the current point is added to the corresponding lane line instance; if the distance is not within the matching threshold range, a new lane line instance is created based on the current point. After all the traversed points have been processed, the lane line instances of each type of lane line in the world coordinate system can be determined.

[0102] In this embodiment, based on the intrinsic and extrinsic parameter information of the image acquisition device, the lane line feature points corresponding to each type of lane line are first transformed into lane line position points in the corresponding world coordinate system. Then, the lane line position points in the world coordinate system corresponding to each type of lane line are clustered based on density to generate lane line instances of each type of lane line in the world coordinate system. By transforming the sampled lane line feature points into lane line instances in 3D coordinates, the accuracy of lane line detection results can be improved.

[0103] As an optional embodiment, based on the intrinsic and extrinsic parameter information of the image acquisition device, the lane feature points corresponding to each type of lane line are converted into lane line position points in the corresponding world coordinate system, including:

[0104] Distortion correction is performed on the lane feature points corresponding to each type of lane line to generate lane line correction points corresponding to each type of lane line.

[0105] Based on the intrinsic and extrinsic parameter information of the image acquisition device, the lane line correction points corresponding to each type of lane line are transformed into lane line position points in the corresponding world coordinate system.

[0106] In this embodiment, the abscissa of the lane line correction point is determined by the following formula 1:

[0107] x c =x(1+k1r) 2 +k2r 4 +k3r 6 )+[2p1xy+p2(r 2 +2x 2 )] Formula 1

[0108] Where, x c y is the abscissa of the lane line correction point after distortion correction, x is the abscissa of the lane line feature point before distortion correction, y is the ordinate of the lane line feature point before distortion correction, k1, k2, and k3 are radial deformation coefficients, p1 and p2 are tangential deformation coefficients, and r is the distance between the lane line feature point and the origin when the center of the optical axis is the origin.

[0109] The ordinate of the lane line correction point is determined using the following formula 2:

[0110] y c =y(1+k1r2 +k2r 4 +k3r 6 )+[2p2xy+p1(r 2 +2y 2 )] Formula 2

[0111] Among them, y c y is the ordinate of the lane line correction point after distortion correction, x is the abscissa of the lane line feature point before distortion correction, y is the ordinate of the lane line feature point before distortion correction, k1, k2, and k3 are radial deformation coefficients, p1 and p2 are tangential deformation coefficients, and r is the distance between the lane line feature point and the origin when the optical axis center is the origin.

[0112] The lane line location in the world coordinate system is determined using the following formula 3:

[0113]

[0114] Where u is the pixel coordinate corresponding to the x-coordinate of the lane line correction point after distortion correction, v is the pixel coordinate corresponding to the y-coordinate of the lane line correction point after distortion correction, r11, r12, r13, r21, r22, r23, r31, r32, r33 are rotation parameters, t1, t2, t3 are translation parameters, Xw, Yw, Zw are the coordinates corresponding to the lane line position points in the world coordinate system, and f x f y , cx, and cy are the internal parameters of the image acquisition device.

[0115] This embodiment corrects distortion of lane feature points corresponding to various lane types, determining the lane correction points for each type of lane. Then, based on the intrinsic and extrinsic parameters of the image acquisition device, these correction points are converted into lane position points in the corresponding world coordinate system. This transforms the sampled lane feature points into lane instances in 3D coordinates, improving the accuracy of lane detection results.

[0116] As an optional embodiment, S105 may specifically include:

[0117] Determine the direction of the third lane line based on the coordinate range information of the lane line instance;

[0118] Determine the start and end points of the lane line based on the direction of the third lane line;

[0119] Based on the lane line location points of each lane line instance, determine the lane line fitting equation.

[0120] In this embodiment, the server determines the coordinate range information of the lane line instance based on the position points of each lane line in the lane line instance, and determines the direction of the third lane line based on the coordinate range information. Then, the start and end points of the lane line are determined along the direction of the third lane line, and the lane line fitting equation is determined by fitting the position points of each lane line in the lane line instance.

[0121] As an example, if there are too many lane line location points in a lane line instance, lane line location points in lane line instances that do not meet the preset fitting conditions can be deleted.

[0122] As an example, when multiple types of lane lines are in the same area, points of single solid lines and single dashed lines can be clustered together, and points of double solid lines and double dashed lines can be clustered together to obtain the same lane line fitting equation. Based on the range of different types of lane line location points in each lane line instance after clustering, the type of lane line can be determined.

[0123] In this embodiment, the direction of the third lane line is determined based on the coordinate range information of the lane line instance. Then, based on the direction of the third lane line, the start and end points of the lane line are determined, and based on the position points of each lane line instance, the lane line fitting equation is determined. This allows for the accurate acquisition of lane line information corresponding to each type of lane line.

[0124] Based on lane line detection methods, this application also provides specific embodiments of lane line detection devices.

[0125] like Figure 4 As shown, the lane line detection device provided in this application embodiment includes an image acquisition module 410, a feature extraction module 420, a connected component analysis module 430, a feature point sampling module 440, and an information generation module 450.

[0126] The image acquisition module 410 is used to acquire a target detection image, which includes at least one lane line and the lane line has a corresponding type label.

[0127] The feature extraction module 420 is used to extract features from the target detection image to obtain the binary map of lane lines corresponding to each type of lane line in the target detection image.

[0128] The connected component analysis module 430 is used to perform connected component analysis on each lane line binary graph based on the type label, and to disconnect the connected components of the lane line binary graph that meet the preset conditions at the intersection points. The preset condition is that the shape of the connected component of the lane line binary graph conforms to the preset target shape.

[0129] The feature point sampling module 440 is used to sample feature points at preset intervals based on the binary images of each lane line after the connected domain is broken at the intersection position, so as to obtain the lane line feature points corresponding to each type of lane line.

[0130] The information generation module 450 is used to generate lane line information corresponding to each type of lane line based on the lane line feature points corresponding to each type of lane line.

[0131] In this embodiment, the image acquisition module 410 acquires a target detection image, which includes at least one lane line, and each lane line has a corresponding type label. Then, the feature extraction module 420 extracts features from the target detection image to obtain binary maps of lane lines corresponding to each type of lane line. Next, the connected component analysis module 430 performs connected component analysis on each lane line binary map based on the type label. If the shape of the connected component in the lane line binary map conforms to a preset target shape, the connected component of the corresponding lane line binary map is broken at the intersection point. Then, the feature point sampling module 440 samples feature points at preset intervals based on each lane line binary map after the connected components are broken at the intersection point, obtaining lane line feature points corresponding to each type of lane line. Finally, the information generation module 450 generates lane line information corresponding to each type of lane line based on the lane line feature points. Thus, this embodiment of the application performs connectivity analysis on the binary graphs of each lane line. If the shape of the connected component of the lane line binary graph conforms to a preset target shape, the connected component of the corresponding lane line binary graph is broken at the intersection point. This effectively avoids the problem of incorrectly fitting a single straight line due to the intersection of lane lines in complex lanes and various types of lane lines, thereby improving the detection accuracy of lane lines in complex lanes and various types of lane lines.

[0132] As an optional embodiment, the feature extraction module 420 specifically includes the following units:

[0133] The feature extraction unit is used to extract features from the target detection image through a semantic segmentation model and generate a lane line mask image.

[0134] The mask extraction unit is used to extract the masks corresponding to each type of lane line in the lane line mask image based on the type label, and generate a binary image of each type of lane line.

[0135] As an optional embodiment, the connected component analysis module 430 specifically includes the following units:

[0136] The connected component analysis unit is used to perform connected component analysis on the binary graphs of each lane line based on the type label, and to determine whether the shape of the connected component corresponding to each lane line binary graph conforms to the preset target shape.

[0137] The pixel zeroing unit is used to assign zeros to each pixel in the lane line binary image corresponding to the intersection point of the connected component with the connected component, based on the intersection point position of the connected component, provided that the shape of the connected component corresponding to the lane line binary image conforms to the preset target shape.

[0138] As an optional embodiment, the connected component analysis unit specifically includes the following sub-units:

[0139] The information acquisition subunit is used to perform connected component analysis on the lane line binary map to obtain the coordinate range information of the first connected component corresponding to the lane line binary map and the connected component type label information.

[0140] The direction determination subunit is used to determine the direction of the first lane line based on the coordinate range information of the first connected domain;

[0141] The frequency determination subunit is used to scan the connected components along the direction of the first lane line and determine the number of times the connected component type label information changes;

[0142] The shape determination subunit is used to determine whether the shape of the connected component corresponding to the lane line binary map conforms to the preset target shape when the number of times the connected component type label information changes exceeds a preset threshold.

[0143] As an optional embodiment, the feature point sampling module 440 specifically includes the following units:

[0144] The information determination unit is used to perform connectivity analysis on the binary graphs of each lane line after the connectivity is broken at the intersection point, and to obtain the coordinate range information of the second connectivity corresponding to the binary graphs of each lane line after the intersection point is broken.

[0145] The direction determination unit is used to determine the direction of the second lane line based on the coordinate range information of the second connected domain;

[0146] The feature point sampling unit is used to scan the connected domain along the direction of the second lane line, and to sample feature points at preset intervals to obtain lane line feature points corresponding to each type of lane line.

[0147] As an optional embodiment, the lane line detection device further includes the following modules:

[0148] The position conversion module is used to convert the lane feature points corresponding to various types of lane lines into lane position points in the corresponding world coordinate system based on the internal and external parameter information of the image acquisition device.

[0149] The location point clustering module is used to cluster the lane line location points in the world coordinate system corresponding to each type of lane line based on density, and generate lane line instances of each type of lane line in the world coordinate system.

[0150] The information generation module 450 is specifically used for:

[0151] Based on lane line instances of each type in the world coordinate system, generate lane line information corresponding to each type of lane line.

[0152] As an optional embodiment, the position conversion module includes the following units:

[0153] The distortion correction unit is used to correct the distortion of lane feature points corresponding to various types of lane lines and generate lane line correction points corresponding to various types of lane lines.

[0154] The position transformation unit is used to transform the lane line correction points corresponding to various types of lane lines into lane line position points in the corresponding world coordinate system based on the intrinsic and extrinsic parameter information of the image acquisition device.

[0155] Figure 5 A schematic diagram of the hardware structure of the lane line detection device provided in an embodiment of this application is shown.

[0156] The lane line detection device may include a processor 501 and a memory 502 storing computer program instructions.

[0157] Specifically, the processor 501 may include a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this application.

[0158] Memory 502 may include mass storage for data or instructions. For example, and not limitingly, memory 502 may include a hard disk drive (HDD), floppy disk drive, flash memory, optical disk, magneto-optical disk, magnetic tape, or Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 502 may include removable or non-removable (or fixed) media. Where appropriate, memory 502 may be internal or external to the integrated gateway disaster recovery device. In a particular embodiment, memory 502 is non-volatile solid-state memory.

[0159] Memory may include read-only memory (ROM), random access memory (RAM), disk storage media devices, optical storage media devices, flash memory devices, and electrical, optical, or other physical / tangible memory storage devices. Therefore, typically, memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the methods according to one aspect of this disclosure.

[0160] The processor 501 implements any of the lane line detection methods in the above embodiments by reading and executing computer program instructions stored in the memory 502.

[0161] In one example, the lane line detection device may also include a communication interface 503 and a bus 510. Wherein, as Figure 5 As shown, the processor 501, memory 502, and communication interface 503 are connected through bus 510 and complete communication with each other.

[0162] The communication interface 503 is mainly used to realize communication between various modules, devices, units and / or equipment in the embodiments of this application.

[0163] Bus 510 includes hardware, software, or both, that couples components of the lane line detection device together. For example, and not limitingly, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a Microchannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or other suitable buses, or combinations of two or more of these. Where appropriate, bus 510 may include one or more buses. Although specific buses are described and illustrated in embodiments of this application, any suitable bus or interconnect is contemplated herein.

[0164] Furthermore, in conjunction with the lane detection methods in the above embodiments, this application embodiment can provide a computer storage medium for implementation. The computer storage medium stores computer program instructions; when these computer program instructions are executed by a processor, they implement any of the lane detection methods in the above embodiments.

[0165] It should be clarified that this application is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of this application is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of this application.

[0166] The functional blocks shown in the above-described structural diagram can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this application are programs or code segments used to perform the required tasks. Programs or code segments can be stored on a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried on a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic media, radio frequency (RF) links, etc. Code segments can be downloaded via computer networks such as the Internet, intranets, etc.

[0167] It should also be noted that the exemplary embodiments mentioned in this application describe methods or systems based on a series of steps or apparatus. However, this application is not limited to the order of the above steps; that is, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.

[0168] The aspects of this disclosure have been described above with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block in the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that these instructions, executable via the processor of the computer or other programmable data processing apparatus, enable the implementation of the functions / actions specified in one or more blocks of the flowchart illustrations and / or block diagrams. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field-programmable logic circuit. It is also understood that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can also be implemented by special-purpose hardware performing the specified functions or actions, or can be implemented by a combination of special-purpose hardware and computer instructions.

[0169] The above description is merely a specific implementation of this application. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, modules, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. It should be understood that the protection scope of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the protection scope of this application.

Claims

1. A lane line detection method, characterized in that, The method includes: Acquire a target detection image, wherein the target detection image includes at least one lane line and the lane line has a corresponding type label; Feature extraction is performed on the target detection image to obtain the lane line binary image corresponding to each type of lane line in the target detection image; Based on the type label, a connected component analysis is performed on each of the lane line binary graphs. The connected components of the lane line binary graphs that meet the preset conditions are broken at the intersection points. The preset conditions are that the shape of the connected component of the lane line binary graph conforms to the preset target shape. Based on the binary map of each lane line after the connected domain is broken at the intersection, feature points are sampled at a preset interval to obtain lane line feature points corresponding to each type of lane line. Based on the lane line feature points corresponding to each type of lane line, lane line information corresponding to each type of lane line is generated. The step of performing connectivity analysis on each lane line binary graph based on the type label, and breaking the connectivity of lane line binary graphs that meet preset conditions at intersection points, includes: Based on the type label, perform connected component analysis on each lane line binary graph to determine whether the shape of the connected component corresponding to each lane line binary graph conforms to a preset target shape; If the shape of the connected component corresponding to the lane line binary image conforms to the preset target shape, then based on the intersection position of the connected component, each pixel in the lane line binary image corresponding to the intersection position of the connected component is assigned zero. The step of performing connected component analysis on each lane line binary graph based on the type label, and determining whether the shape of the connected component corresponding to each lane line binary graph conforms to a preset target shape, includes: Perform connected component analysis on the lane line binary graph to obtain the label, coordinate range, and area information of the connected component corresponding to the lane line binary graph, and determine the lane line direction based on the coordinate range of the connected component; When the lane line direction is biased towards the x-axis, the connected region is scanned along the x-axis and sampled according to a preset scanning interval; When the lane line direction is biased towards the y-axis, the connected region is scanned along the y-axis and sampled according to a preset scanning interval; Based on the sampling points obtained from the sampling, determine whether there is a change in the connected component label within the range of the connected component corresponding to the current scanning axis; If the connected component label changes three times, then at least two points belonging to this connected component on the scan axis are broken in the middle, and the connected component state at this time is recorded as "broken". If the number of times the connected component label changes is less than three, then there is no phenomenon of two points in the same connected component being disconnected in the middle on the scan axis at this time, and the state of the connected component at this time is recorded as "not disconnected"; If the connected component state changes between "disconnected" and "not disconnected" during the scanning process, the connected component is Y-shaped or V-shaped, and the location of the change is the location of the intersection. The shape of the connected component corresponding to the lane line binary map conforms to the preset target shape. If the connected region state does not change between "disconnected" and "not disconnected" during the scanning process, then the connected region is not Y-shaped or V-shaped, and the shape of the connected region corresponding to the lane line binary map does not conform to the preset target shape.

2. The method according to claim 1, characterized in that, The step of extracting features from the target detection image to obtain binary lane maps corresponding to each type of lane line in the target detection image includes: Feature extraction is performed on the target detection image using a semantic segmentation model to generate a lane line mask image; Based on the type labels, the masks corresponding to each type of lane line in the lane line mask image are extracted to generate a binary map of each type of lane line.

3. The method according to any one of claims 1-2, characterized in that, The binary images of each lane line after the connected components are broken at the intersection points are used to sample feature points at preset intervals to obtain lane line feature points corresponding to each type of lane line, including: Connectivity analysis is performed on the binary graphs of each lane line after the connected component is broken at the intersection point to obtain the coordinate range information of the second connected component corresponding to the binary graphs of each lane line after the intersection point is broken. Based on the coordinate range information of the second connected domain, the direction of the second lane line is determined; Scan the connected domain along the direction of the second lane line, and sample feature points at preset intervals to obtain lane line feature points corresponding to each type of lane line.

4. The method according to any one of claims 1-2, characterized in that, After sampling feature points at preset intervals based on the binary images of each lane line after the connected components are broken at intersection points to obtain the lane line feature points corresponding to each type of lane line, the method further includes: Based on the intrinsic and extrinsic parameter information of the image acquisition device, the lane feature points corresponding to each type of lane line are transformed into lane position points in the corresponding world coordinate system. The lane line location points in the world coordinate system corresponding to each type of lane line are clustered based on density to generate lane line instances of each type of lane line in the world coordinate system. The process of generating lane line information corresponding to each type of lane line based on the lane line feature points includes: Based on lane line instances of each type of lane line in the world coordinate system, lane line information corresponding to each type of lane line is generated.

5. The method according to claim 4, characterized in that, The process of converting lane feature points corresponding to each type of lane line into lane position points in the corresponding world coordinate system based on the intrinsic and extrinsic parameter information of the image acquisition device includes: Distortion correction is performed on the lane line feature points corresponding to each type of lane line to generate lane line correction points corresponding to each type of lane line. Based on the intrinsic and extrinsic parameter information of the image acquisition device, the lane line correction points corresponding to each type of lane line are transformed into lane line position points in the corresponding world coordinate system.

6. A lane line detection device, characterized in that, The device includes: An image acquisition module is used to acquire a target detection image, wherein the target detection image includes at least one lane line and the lane line has a corresponding type label; The feature extraction module is used to extract features from the target detection image to obtain a binary image of lane lines corresponding to each type of lane line in the target detection image. The connected component analysis module is used to perform connected component analysis on each of the lane line binary graphs based on the type label, and to disconnect the connected components of the lane line binary graphs that meet the preset conditions at the intersection points. The preset conditions are that the shape of the connected component of the lane line binary graph conforms to the preset target shape. The feature point sampling module is used to sample feature points at preset intervals based on the binary images of each lane line after the connected domain is broken at the intersection position, so as to obtain the lane line feature points corresponding to each type of lane line. The information generation module is used to generate lane line information corresponding to each type of lane line based on the lane line feature points corresponding to each type of lane line. The connected component analysis module is further configured to perform connected component analysis on each lane line binary image based on the type label, determine whether the shape of the connected component corresponding to each lane line binary image conforms to a preset target shape; if the shape of the connected component corresponding to the lane line binary image conforms to the preset target shape, assign zeros to each pixel point corresponding to the intersection point of the connected component in the lane line binary image based on the intersection point position of the connected component; perform connected component analysis on the lane line binary image to obtain the label, coordinate range, and area information of the connected component corresponding to the lane line binary image, and determine the lane line direction based on the coordinate range of the connected component; if the lane line direction is biased towards the x-axis, scan the connected component along the x-axis and sample according to a preset scanning interval; if the lane line direction is biased towards the y-axis, scan the connected component along the y-axis and sample according to a preset scanning interval; determine the current scanning axis based on the sampled points. Within the scope of the connected region, are there any changes in the connected region label? If the number of times the connected region label changes reaches three, then at least two points belonging to this connected region on the scanning axis are broken in the middle, and the connected region state is recorded as "broken". If the number of times the connected region label changes is less than three, then there are no two points belonging to this connected region broken in the middle on the scanning axis, and the connected region state is recorded as "not broken". If the connected region state changes between "broken" and "not broken" during the scanning process, then this connected region is Y-shaped or V-shaped, and the location of the change is the location of the intersection. The shape of the connected region corresponding to the lane line binary map conforms to the preset target shape. If the connected region state does not change between "broken" and "not broken" during the scanning process, then this connected region is not Y-shaped or V-shaped, and the shape of the connected region corresponding to the lane line binary map does not conform to the preset target shape.

7. An electronic device, characterized in that, The device includes: a processor and a memory storing computer program instructions; When the processor executes the computer program instructions, it implements the lane line detection method as described in any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer program instructions that, when executed by a processor, implement the lane line detection method as described in any one of claims 1-5.

Images