Lane line detection method and device, computer device and storage medium

Abstract

This application relates to the fields of autonomous driving and image processing technology, and in particular to a lane line detection method, apparatus, computer device, and storage medium. The method includes: extracting features from a lane line image to obtain lane line features; determining target pixels containing lane lines from the lane line features; inputting the lane line features into a position detection head network to obtain the intersection position of the lane line and the corresponding boundary line of the target pixel; and determining the position information of the lane line in the target pixel based on the intersection position of the lane line and the corresponding boundary line. This application can improve lane detection accuracy.

Description

Technical Field

[0001] This application relates to the fields of autonomous driving and image processing technology, and in particular to a lane line detection method, apparatus, computer device, and storage medium. Background Technology

[0002] With the development of artificial intelligence and deep learning technologies in recent years, deep learning-based lane detection methods have gradually replaced traditional lane detection methods. Lane detection differs from ordinary object detection because lane lines have special shapes, significant aspect ratios, and occupy a large area in the image.

[0003] Traditional lane detection algorithms are generally based on edge detection, Hough transform, and straight-line / curve fitting.

[0004] Traditional lane detection algorithms perform poorly when driving on curves, and their accuracy in practical applications needs improvement. Therefore, improvements are urgently needed. Summary of the Invention

[0005] Therefore, it is necessary to provide a lane line detection method, device, computer equipment, and storage medium that can improve lane detection accuracy in response to the above-mentioned technical problems.

[0006] Firstly, this application provides a lane line detection method, the method comprising:

[0007] Feature extraction is performed on the lane line image to obtain lane line features;

[0008] Based on lane line features, target pixels containing lane lines are determined from the lane line image;

[0009] The lane line features are input into the location detection head network to obtain the intersection point of the lane line and the corresponding boundary line of the target pixel.

[0010] The position information of the lane line in the target pixel is determined by the intersection point of the lane line in the target pixel and the corresponding boundary line of the target pixel.

[0011] In one embodiment, the position information of the lane line in the target pixel is determined based on the intersection point of the lane line in the target pixel and the corresponding boundary line of the target pixel, including:

[0012] Based on the intersection points of the lane lines and the corresponding boundary lines of the target pixels, a connecting line is formed between the intersection points;

[0013] The position information of the lane line in the target pixel is determined based on the midpoint of the connecting line.

[0014] In one embodiment, determining target pixels containing lane lines from a lane line image based on lane line features includes:

[0015] The lane line features are input into the confidence detection head network, and the target pixels containing the lane lines are determined from the lane line image based on the lane line features.

[0016] In one embodiment, the method further includes:

[0017] The lane line features are input into the category detection head network to obtain the category of the lane line in the target pixel.

[0018] In one embodiment, the method further includes:

[0019] The lane line features are input into the number detection head network to obtain the number of lane lines in the lane line image.

[0020] In one embodiment, feature extraction is performed on the lane line image to obtain lane line features, including:

[0021] The lane line image is input into the initial feature extraction module in the feature extraction network to obtain the initial image features of the lane line image;

[0022] The initial image features are input into the multi-scale fusion module in the feature extraction network to obtain lane line features.

[0023] In one embodiment, the multi-scale fusion module includes multiple separable convolutional layers and feature selection layers of different scales; the initial image features are input into the multi-scale fusion module in the feature extraction network to obtain lane line features, including:

[0024] The initial image features are input into each separable convolutional layer to obtain features at different scales;

[0025] Image features at different scales are input into the feature selection layer to obtain lane line features.

[0026] In one embodiment, the method further includes:

[0027] Obtain the original image containing lane lines;

[0028] The original image is inversely transformed using an inverse perspective transformation subnetwork to obtain a lane line image.

[0029] Secondly, this application also provides a lane line detection device, which includes:

[0030] The feature extraction module is used to extract features from the lane line image to obtain lane line features;

[0031] The target detection module is used to determine target pixels containing lane lines from the lane line image based on lane line features.

[0032] The intersection detection module is used to input lane line features into the position detection head network to obtain the intersection position of the lane line and the corresponding boundary line of the target pixel.

[0033] The position determination module is used to determine the position information of the lane line in the target pixel based on the intersection point of the lane line in the target pixel and the corresponding boundary line of the target pixel.

[0034] Thirdly, this application also provides a computer device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0035] Feature extraction is performed on the lane line image to obtain lane line features;

[0036] Based on lane line features, target pixels containing lane lines are determined from the lane line image;

[0037] The lane line features are input into the location detection head network to obtain the intersection point of the lane line and the corresponding boundary line of the target pixel.

[0038] The position information of the lane line in the target pixel is determined by the intersection point of the lane line in the target pixel and the corresponding boundary line of the target pixel.

[0039] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the following steps:

[0040] Feature extraction is performed on the lane line image to obtain lane line features;

[0041] Based on lane line features, target pixels containing lane lines are determined from the lane line image;

[0042] The lane line features are input into the location detection head network to obtain the intersection point of the lane line and the corresponding boundary line of the target pixel.

[0043] The position information of the lane line in the target pixel is determined by the intersection point of the lane line in the target pixel and the corresponding boundary line of the target pixel.

[0044] Fifthly, this application also provides a computer program product comprising a computer program that, when executed by a processor, performs the following steps:

[0045] Feature extraction is performed on the lane line image to obtain lane line features;

[0046] Based on lane line features, target pixels containing lane lines are determined from the lane line image;

[0047] The lane line features are input into the location detection head network to obtain the intersection point of the lane line and the corresponding boundary line of the target pixel.

[0048] The position information of the lane line in the target pixel is determined by the intersection point of the lane line in the target pixel and the corresponding boundary line of the target pixel.

[0049] The aforementioned lane line detection method, apparatus, computer equipment, and storage medium can identify any pixel in a lane line image as a target pixel if the lane line passes through it. When the lane line passes through a target pixel, it intersects with the boundary line of the target pixel. The position of the intersection point is obtained through a position detection head network. Then, based on the position of the intersection point, the position information of the lane line within the target pixel is determined. By combining the position information of the lane lines in each target pixel, the distribution of the lane lines in the lane line image can be obtained. In contrast, traditional lane line detection techniques select only one pixel in each row of pixels in the lane line image that the lane line passes through. This only detects lane lines in the vertical or near-vertical direction. The detection effect is not ideal for horizontal lane lines, especially lane lines with large curvatures. The detection effect is poor, especially when vehicles are turning or making U-turns at intersections. Therefore, this application has higher detection accuracy for circular lane lines. Attached Figure Description

[0050] Figure 1 This is a flowchart illustrating a lane line detection method in one embodiment;

[0051] Figure 2 This is a schematic diagram of lane lines and target pixels in one embodiment;

[0052] Figure 3 This is a flowchart illustrating the process of determining the position information of a lane line in a target pixel in one embodiment.

[0053] Figure 4 This is a flowchart illustrating the process of obtaining lane line features in one embodiment;

[0054] Figure 5 This is a schematic diagram of the network structure of a multi-scale fusion module in one embodiment;

[0055] Figure 6 This is a flowchart illustrating the lane detection model in another embodiment;

[0056] Figure 7This is a structural block diagram of a lane line detection device in one embodiment;

[0057] Figure 8 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0058] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0059] As autonomous driving technology matures, many technologies have reached mass production requirements and are ready for practical road applications. In autonomous driving systems, lane detection algorithms provide perception information to the autonomous driving system during vehicle operation. This is a key technology ensuring vehicles stay within designated lanes and comply with traffic rules. It is also crucial for subsequent path planning by the intelligent driving system; its reliability and accuracy are paramount to the safety and stability of autonomous vehicles.

[0060] With the development of artificial intelligence and deep learning technologies in recent years, deep learning-based lane detection methods have gradually replaced traditional lane detection methods. Traditional lane detection algorithms are generally based on edge detection, Hough transform, and straight-line curve fitting, but these methods are less effective in detecting lanes when driving on curves.

[0061] In one embodiment, such as Figure 1 As shown, a lane line detection method is provided. Taking the application of this method to a computer device as an example, the method includes the following steps:

[0062] S101, extract features from the lane line image to obtain lane line features.

[0063] The lane line image refers to an image of the lane lines at the location of the vehicle, captured by an image acquisition device. Optionally, the image acquisition device can be a camera, and the device can be mounted on a fixed object beside the road, or at the front or rear of the vehicle.

[0064] Specifically, the lane line features in this embodiment may include horizontal lines, straight lines, and intersections of straight lines and horizontal lines in the lane line image. The purpose of analyzing these lane line features is to retain the pixels related to the lane lines in the lane line image and the features of those pixels. Optionally, a deep learning algorithm can be used to extract features from the lane line image. Furthermore, before feature extraction, the lane line image can be scaled to obtain a lane line image of a preset size. This scaling operation reduces the computational efficiency of image feature extraction and minimizes unnecessary computational processes.

[0065] S102, Based on the lane line features, determine the target pixel points containing the lane lines from the lane line image.

[0066] Specifically, deep learning algorithms can be further used to analyze the lane line features, including horizontal lines, straight lines, and intersections of straight lines and horizontal lines, in the lane line images to determine the target pixels containing lane lines from the lane line images.

[0067] It is understandable that there may be multiple lane lines in a lane line image. Therefore, in one possible approach, a pixel containing any lane line can be used as the target pixel. The first step is to separate the lane lines from the background, and then the target pixels corresponding to each lane line are distinguished. In another possible approach, when determining the target pixel, the confidence information of which lane line the target pixel belongs to is analyzed simultaneously, thereby distinguishing the target pixels corresponding to the lane lines.

[0068] S103, input the lane line features into the position detection head network to obtain the intersection position of the lane line and the corresponding boundary line of the target pixel.

[0069] Specifically, when a lane line passes through a target pixel, it must intersect the target pixel's boundary (one of the four boundary lines) at least once. For example... Figure 2 As shown, this embodiment focuses on the intersections in the lane line image, and by analyzing the positions of the intersections through a position detection head network, it can ensure that no pixel points along the lane line are missed.

[0070] Specifically, the position detection head network can be a network formed by convolution, which can output the offset of the intersection point position, that is, the horizontal offset (x coordinate) and vertical offset (y coordinate) of each intersection point.

[0071] S104, determine the position information of the lane line in the target pixel based on the intersection point of the lane line in the target pixel and the corresponding boundary line of the target pixel.

[0072] Understandably, if a lane line intersects the boundary of a target pixel at two points, and the x-coordinates or y-coordinates of the two points are the same, then the lane line is likely a vertical or horizontal lane line. If the lane line intersects the boundary of a target pixel at two points, and the x-coordinates or y-coordinates of the two points are the same, then the lane line is likely a circular lane line. Therefore, by analyzing the intersection points of the lane line with the corresponding boundary line of the target pixel, the distribution of lane lines can be determined.

[0073] In the lane detection method described above, when a lane line passes through a target pixel, it intersects with the boundary line of the target pixel. The position of the intersection is obtained through a position detection head network. Then, based on the position of the intersection, the position information of the lane line within the target pixel is determined. By combining the position information of the lane lines in each target pixel, the distribution of the lane lines in the lane line image can be obtained. Traditional lane detection techniques select only one pixel in each row of pixels in the lane line image that the lane line passes through. Therefore, they can only detect lane lines in the vertical or near-vertical direction. The detection effect is not ideal for horizontal lane lines, or even lane lines with large curvatures. In particular, the detection effect is poor when vehicles are turning or making U-turns at intersections. In this application, as long as the lane line passes through any pixel in the lane line image, that pixel can be identified as a target pixel. By analyzing each target pixel, the distribution of lane lines (including circular, vertical, or horizontal states) can be accurately analyzed. Therefore, this application has higher detection accuracy for circular lane lines.

[0074] In traditional technologies, instance-based segmentation algorithms often have large models and require complex post-processing steps for lane line fitting. In autonomous driving systems, this puts significant pressure on resource consumption and makes it difficult to meet real-time requirements. Therefore, it is necessary to use more lightweight networks and structured model outputs to reduce post-processing steps and computational consumption. Figure 3 As shown, this embodiment provides an optional method for determining the position information of the lane line in a target pixel based on the intersection point of the lane line in the target pixel and the corresponding boundary line of the target pixel, that is, a method for refining S104. The specific implementation process may include:

[0075] S301, Based on the intersection point of the lane line in the target pixel and the corresponding boundary line of the target pixel, form a connecting line between the intersection points.

[0076] It is understandable that, for the same lane line, the distribution of the lane line can be obtained by connecting the target pixels corresponding to the lane line. Furthermore, in order to analyze the specific location of the lane line and reduce the amount of calculation, the embodiment of this application approximates the distribution of the lane line in the target pixel as a straight line, that is, according to the intersection position of the lane line in the target pixel and the corresponding boundary line of the target pixel, a connecting line is formed between the intersection points.

[0077] S302, determine the position information of the lane line in the target pixel based on the midpoint of the connecting line.

[0078] Specifically, based on the midpoint of the connecting line, the position information of the lane line in the target pixel is determined, thus obtaining the position point of the lane line when it passes through each target pixel, i.e., the coordinate value (x, y).

[0079] In this embodiment, by forming connecting lines between intersections and determining the midpoint of the connecting lines as the position of the lane line within the target pixel, the position of the lane line can be accurately located while reducing the amount of computation.

[0080] It is understandable that lane line detection differs from ordinary object detection. Lane lines have unique shapes, significant aspect ratios, and occupy a large area in the image, posing challenges in network structure design, regression methods, and loss design. Therefore, improving the network structure and adopting appropriate detection heads and regression methods tailored to the structural characteristics of lane lines is essential. Thus, in one embodiment, this embodiment provides an optional method for determining target pixels containing lane lines from a lane line image based on lane line features, i.e., a method for refining S102. The specific implementation process may include: inputting lane line features into a confidence detection head network, and determining target pixels containing lane lines from the lane line image based on the lane line features.

[0081] Specifically, in this embodiment, when analyzing lane line features, the confidence detection head network determines that if a lane line passes through a certain pixel, the confidence level of that pixel is 1, and it is considered a target pixel. Therefore, lane lines of any shape can be detected through each target pixel.

[0082] Furthermore, the lane line detection method also includes: inputting lane line features into a category detection head network to obtain the category of the lane line in the target pixel.

[0083] Specifically, the category detection head network uses a clustering post-processing method to determine which lane a target pixel belongs to. The clustering methods typically employed include mean shift and hierarchical clustering, which are characterized by not requiring the specified number of categories.

[0084] Furthermore, the lane line detection method also includes: inputting lane line features into a number detection head network to obtain the number of lane lines in the lane line image.

[0085] Specifically, the number of lane lines is regressed by the count head network, and the number of lane lines is also the number of categories in the clustering algorithm of the above-mentioned category head network.

[0086] In this embodiment, the accuracy of lane line detection is further improved by adding a category detection head network and a number detection head network.

[0087] In one embodiment, obtaining a lane line image includes: obtaining an original image containing lane lines; and performing an inverse perspective transformation on the original image based on an inverse perspective transformation subnetwork to obtain the lane line image.

[0088] Understandably, in an autonomous driving system, the surround view system provides autonomous vehicles with environmental information around the parking space, especially the near side. The surround view vision system is generally composed of four fisheye cameras: front, rear, left, and right. Compared with ordinary cameras, fisheye cameras have advantages such as a large field of view and a long shooting distance. Therefore, the original image in this embodiment can be the images captured by the fisheye cameras.

[0089] Specifically, performing inverse perspective transformation on the lane line image based on the inverse perspective transformation sub-network means stitching together the four fisheye cameras (IPM) to obtain a complete AVM (Around View Monitor) top view as the lane line image.

[0090] Furthermore, this embodiment provides an optional method for extracting features from a lane line image to obtain lane line features, that is, it provides a method for refining S101. For example... Figure 4 As shown, the specific implementation process may include:

[0091] S401, input the lane line image into the initial feature extraction module in the feature extraction network to obtain the initial image features of the lane line image.

[0092] Specifically, the initial feature extraction module is a ConvNeXt network. ConvNeXt networks have different structures such as base, small, and tiny. The tiny model has fewer parameters, making it more suitable for deployment on mobile devices. In this implementation, the tiny structure was further optimized by reducing the number of network channels through channel pruning. The ConvNeXt network outperforms the Swin-T model in multiple classification and recognition tasks, achieving the best performance. Therefore, in this embodiment, ConvNeXt_tiny was specifically used as the backbone network for the lane detection task, achieving satisfactory results in both detection performance and inference speed.

[0093] Furthermore, after extracting the initial image features using the ConvNeXt network, the multi-scale fusion module employs an attention-based multi-scale fusion method. Compared to ordinary fusion methods like FPN and PANnet, this method is simpler and lighter, and it introduces an attention mechanism for automatic selection of scale information. The specific process is as follows:

[0094] S402, input the initial image features into the multi-scale fusion module in the feature extraction network to obtain lane line features.

[0095] Among them, such as Figure 5As shown, the multi-scale fusion module includes multiple separable convolutional layers and feature selection layers of different scales; correspondingly, the initial image features are input into the multi-scale fusion module in the feature extraction network to obtain lane line features, including: inputting the initial image features into each separable convolutional layer to obtain features of different scales; and inputting the image features of different scales into the feature selection layer to obtain lane line features.

[0096] Understandably, lane line structures differ from ordinary target structures. Most lane lines are elongated structures with a large aspect ratio, and most lane lines extend from the bottom to the top of the image. Therefore, lane line detection requires a large receptive field. The usual approach is to increase the size of the convolutional kernel. However, excessively large convolutional kernels mean more computation and resource consumption. Therefore, this embodiment uses a depth-wise approach based on the attention mechanism's multi-scale fusion sub-network. It extracts features at different scales through three sets of separable convolutions (multi-scale features) of different scales. At the same time, each scale is implemented using two separable convolutions of different dimensions.

[0097] Optionally, this embodiment uses three sets of convolutional kernel scales of 7, 13, and 19, which maximizes the reduction of computational load while obtaining a large receptive field. Furthermore, after extracting features at different scales, the features at the three scales are concatenated with the original features, and then a 1*1 feature selection layer (channel attention) is used for feature selection for attention.

[0098] In this embodiment, the multi-scale fusion module uses separable convolutional groups composed of large convolutional kernels, which increases the receptive field of the model and makes the model more suitable for quantitative regression. Furthermore, the convolution is optimized based on the lane line shape, thereby reducing the computational load.

[0099] For example, based on the above embodiments, this embodiment provides a lane line detection network model applied to the above lane line detection method, such as... Figure 6 As shown, this lane detection network model includes a feature extraction network and a detection head network.

[0100] The feature extraction network includes an initial feature extraction module 61 and a multi-scale fusion module 62; the detection head network includes a confidence detection head network 63, a location detection head network 64, a category detection head network 65, and a number detection head network 66.

[0101] The initial feature extraction module 61 is a ConvNeXt_tiny backbone subnetwork used to extract initial image features of lane line images. The multi-scale fusion module 62 includes multiple separable convolutional layers and feature selection layers of different scales, specifically used to: input the initial image features into each separable convolutional layer to obtain features of different scales; and input the image features of different scales into the feature selection layer to obtain lane line features.

[0102] Specifically, the confidence detection head network 63 is used to determine the target pixel containing the lane line from the lane line image based on the lane line features; the position detection head network 64 is used to regress the intersection position of the lane line and the corresponding boundary line of the target pixel; and to determine the position information of the lane line in the target pixel based on the intersection position of the lane line and the corresponding boundary line of the target pixel; the category detection head network 65 is used to regress the category of the lane line in the target pixel; and the number detection head network 66 is used to regress the number of lane lines in the lane line image.

[0103] The lane detection network model described above employs a multi-scale fusion module 62 composed of separable convolutions, balancing computational efficiency with large-scale convolutions, and improving detection performance for lane lines of different shapes, including horizontal, vertical, and inclined lane lines. Furthermore, compared to conventional row-wise keypoint regression methods, which can only detect lane lines in the vertical or near-vertical direction, the above lane detection network model optimizes the regression method for target pixels using a confidence detection head network 63, a position detection head network 64, a category detection head network 65, and a count detection head network 66. If a lane line passes through a pixel, the confidence of that pixel is set to 1. The regression method of the position detection network is also improved, enabling the model to detect lane lines of arbitrary shapes. A lane line count regression branch is added to improve the accuracy of lane line category regression.

[0104] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0105] Based on the same inventive concept, this application also provides a lane line detection device for implementing the lane line detection method described above. The solution provided by this device is similar to the implementation described in the above method; therefore, the specific limitations of one or more lane line detection device embodiments provided below can be found in the limitations of the lane line detection method described above, and will not be repeated here.

[0106] In one embodiment, such as Figure 7 As shown, a lane line detection device 1 is provided, including: a feature extraction module 11, a target detection module 12, an intersection detection module 13, and a position determination module 14, wherein:

[0107] Feature extraction module 11 is used to extract features from the lane line image to obtain lane line features;

[0108] The target detection module 12 is used to determine target pixels containing lane lines from the lane line image based on lane line features.

[0109] Intersection detection module 13 is used to input lane line features into the position detection head network to obtain the intersection position of the lane line and the corresponding boundary line of the target pixel.

[0110] The position determination module 14 is used to determine the position information of the lane line in the target pixel based on the intersection position of the lane line in the target pixel and the corresponding boundary line of the target pixel.

[0111] In one embodiment, the intersection detection module 13 is further configured to: form a connecting line between intersection points based on the intersection position of the lane line in the target pixel and the corresponding boundary line of the target pixel;

[0112] The position information of the lane line in the target pixel is determined based on the midpoint of the connecting line.

[0113] In one embodiment, the target detection module 12 is further configured to: input lane line features into a confidence detection head network, and determine target pixels containing lane lines from the lane line image based on the lane line features.

[0114] In one embodiment, the lane line detection device further includes a category detection module, which is used to: input lane line features into a category detection head network to obtain the category of the lane line in the target pixel.

[0115] In one embodiment, the lane line detection device further includes a count detection module, which is used to: input the target pixel into the count detection head network to obtain the number of lane lines in the target pixel.

[0116] In one embodiment, the feature extraction module 11 includes:

[0117] The initial feature submodule is used to input the lane line image into the initial feature extraction module 11 in the feature extraction network to obtain the initial image features of the lane line image;

[0118] The multi-scale fusion submodule is used to input the initial image features into the multi-scale fusion module in the feature extraction network to obtain lane line features.

[0119] In one embodiment, the multi-scale fusion module includes multiple separable convolutional layers and feature selection layers of different scales; the multi-scale fusion submodule is further configured to: input the initial image features into each separable convolutional layer to obtain features of different scales; and input the image features of different scales into the feature selection layer to obtain lane line features.

[0120] In one embodiment, the lane line detection module further includes a transformation module, which is used to: acquire an original image containing lane lines; and perform an inverse perspective transformation on the original image based on an inverse perspective transformation subnetwork to obtain a lane line image.

[0121] Each module in the aforementioned lane line detection device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0122] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 8 As shown, the computer device includes a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When executed by the processor, the computer program implements a lane detection method. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the computer device casing, or an external keyboard, touchpad, or mouse.

[0123] Those skilled in the art will understand that Figure 8The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0124] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0125] Feature extraction is performed on the lane line image to obtain lane line features;

[0126] Based on lane line features, target pixels containing lane lines are determined from the lane line image;

[0127] The lane line features are input into the location detection head network to obtain the intersection point of the lane line and the corresponding boundary line of the target pixel.

[0128] The position information of the lane line in the target pixel is determined by the intersection point of the lane line in the target pixel and the corresponding boundary line of the target pixel.

[0129] In one embodiment, when the processor executes the logic of a computer program to determine the position information of a lane line in a target pixel based on the intersection position of the lane line in the target pixel and the corresponding boundary line of the target pixel, the following steps are specifically implemented: forming a connecting line between the intersection points based on the intersection position of the lane line in the target pixel and the corresponding boundary line of the target pixel; determining the position information of the lane line in the target pixel based on the midpoint of the connecting line.

[0130] In one embodiment, when the processor executes the logic of a computer program to determine target pixels containing lane lines from a lane line image based on lane line features, it specifically implements the following steps: inputting lane line features into a confidence detection head network, and determining target pixels containing lane lines from the lane line image based on the lane line features.

[0131] In one embodiment, when the processor executes the computer program, it also performs the following steps: inputting lane line features into a category detection head network to obtain the category of the lane line in the target pixel.

[0132] In one embodiment, when the processor executes the computer program, it also performs the following steps: inputting lane line features into a count detection head network to obtain the number of lane lines in the lane line image.

[0133] In one embodiment, when the processor executes a computer program to extract features from a lane line image and obtains lane line features, the following steps are specifically implemented: inputting the lane line image into the initial feature extraction module in the feature extraction network to obtain the initial image features of the lane line image; inputting the initial image features into the multi-scale fusion module in the feature extraction network to obtain the lane line features.

[0134] In one embodiment, the multi-scale fusion module includes multiple separable convolutional layers and feature selection layers of different scales; in one embodiment, when the processor executes a computer program to input initial image features into the multi-scale fusion module in the feature extraction network to obtain lane line features, the following steps are specifically implemented: inputting the initial image features into each separable convolutional layer to obtain features of different scales; inputting the image features of different scales into the feature selection layer to obtain lane line features.

[0135] In one embodiment, when the processor executes the computer program, it further performs the following steps: acquiring an original image containing lane lines; performing an inverse perspective transformation on the original image based on an inverse perspective transformation subnetwork to obtain a lane line image.

[0136] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0137] Feature extraction is performed on the lane line image to obtain lane line features;

[0138] Based on lane line features, target pixels containing lane lines are determined from the lane line image;

[0139] The lane line features are input into the location detection head network to obtain the intersection point of the lane line and the corresponding boundary line of the target pixel.

[0140] The position information of the lane line in the target pixel is determined by the intersection point of the lane line in the target pixel and the corresponding boundary line of the target pixel.

[0141] In one embodiment, when the logic of a computer program determining the position information of a lane line in a target pixel based on the intersection position of the lane line in the target pixel with the corresponding boundary line is executed by the processor, the following steps are specifically implemented: forming a connecting line between the intersection points based on the intersection position of the lane line in the target pixel with the corresponding boundary line; determining the position information of the lane line in the target pixel based on the midpoint of the connecting line.

[0142] In one embodiment, when the logic of a computer program determining target pixels containing lane lines from a lane line image based on lane line features is executed by a processor, the specific steps are as follows: inputting lane line features into a confidence detection head network, and determining target pixels containing lane lines from the lane line image based on the lane line features.

[0143] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: inputting lane line features into a category detection head network to obtain the category of the lane line in the target pixel.

[0144] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: inputting lane line features into a count detection head network to obtain the number of lane lines in the lane line image.

[0145] In one embodiment, when the logic of a computer program extracting features from a lane line image and obtaining lane line features is executed by a processor, the following steps are specifically implemented: inputting the lane line image into the initial feature extraction module in the feature extraction network to obtain the initial image features of the lane line image; inputting the initial image features into the multi-scale fusion module in the feature extraction network to obtain the lane line features.

[0146] In one embodiment, the multi-scale fusion module includes multiple separable convolutional layers and feature selection layers of different scales. When the logic of the computer program inputting initial image features into the multi-scale fusion module in the feature extraction network to obtain lane line features is executed by the processor, the following steps are specifically implemented: inputting the initial image features into each separable convolutional layer to obtain features of different scales; inputting the image features of different scales into the feature selection layer to obtain lane line features.

[0147] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: acquiring an original image containing lane lines; and performing an inverse perspective transformation on the original image based on an inverse perspective transformation subnetwork to obtain a lane line image.

[0148] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0149] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0150] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A lane line detection method characterized by, The method includes: The lane line image is input into the initial feature extraction module in the feature extraction network to obtain the initial image features of the lane line image; The initial image features are input into multiple separable convolutional layers of different scales in the multi-scale fusion module of the feature extraction network to obtain features at different scales. Image features at different scales are input into the feature selection layer of the multi-scale fusion module to obtain lane line features; The lane line features are input into the confidence detection head network, and the target pixels containing lane lines are determined from the lane line image based on the lane line features. The lane line features are input into the position detection head network to obtain the intersection position of the lane line and the corresponding boundary line of the target pixel. Based on the intersection points of the lane lines in the target pixels and the corresponding boundary lines of the target pixels, a connecting line is formed between the intersection points; The position information of the lane line in the target pixel is determined based on the midpoint of the connecting line.

2. The method of claim 1, wherein, The method further includes: The lane line features are input into the category detection head network to obtain the category of the lane line in the target pixel.

3. The method of claim 1, wherein, The method further includes: The lane line features are input into a count detection head network to obtain the number of lane lines in the lane line image; wherein, the count detection head network is a neural network used to regress the number of lane lines in the lane line image.

4. The method of claim 1, wherein, The method further includes: Obtain the original image containing lane lines; The original image is subjected to inverse perspective transformation based on the inverse perspective transformation sub-network to obtain the lane line image.

5. The method of claim 1, wherein, The lane line image is an image of the lane line at the location of the vehicle, acquired by an image acquisition device. The image acquisition device is located on a fixed object beside the road where the vehicle is located, at the front of the vehicle, or at the rear of the vehicle.

6. The method of claim 1, wherein, The lane line features include horizontal lines, straight lines, and the intersections of straight lines and horizontal lines in the lane line image.

7. A lane line detection device characterized by comprising: The device includes: The feature extraction module is used to input the lane line image into the initial feature extraction module in the feature extraction network to obtain the initial image features of the lane line image; input the initial image features into multiple separable convolutional layers of different scales in the multi-scale fusion module in the feature extraction network to obtain features of different scales; input the image features of different scales into the feature selection layer in the multi-scale fusion module to obtain lane line features; The target detection module is used to input the lane line features into the confidence detection head network, and determine the target pixels containing the lane lines from the lane line image based on the lane line features. The intersection detection module is used to input the lane line features into the position detection head network to obtain the intersection position of the lane line and the corresponding boundary line of the target pixel. The position determination module is used to form a connecting line between the intersection points of the lane line in the target pixel and the corresponding boundary line of the target pixel; and to determine the position information of the lane line in the target pixel based on the midpoint of the connecting line.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images