A lane line detection method based on starting point guidance
By introducing a starting point detection head and starting point coordinates to enhance the hierarchical fusion feature in lane line detection, the problems of insufficient flexibility and quality of line anchors are solved, and lane line detection with higher accuracy and robustness is achieved.
Patent Information
- Application Number
- CN202410704054.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-06-03
- Publication Date
- 2025-11-07
- Estimated Expiration
- 2044-06-03
AI Technical Summary
Existing lane line detection methods based on line anchors lack flexibility, have low-quality line anchors, and are difficult to adapt to complex lane line types, resulting in insufficient detection accuracy and robustness.
A starting point detection head is introduced to filter line anchors by predicting the starting point of the lane line, thereby reducing the number of line anchors and improving their quality. The hierarchical fusion feature is enhanced by combining the starting point coordinates to improve the accuracy and robustness of lane line detection.
By using a starting point-guided method, lane line spatial features can be accurately captured, redundant starting points can be reduced, line anchor quality can be improved, detection performance can be enhanced, and the accuracy and robustness of lane line detection can be increased.
Smart Images
Figure CN118552924B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of computer vision, and particularly relates to a lane line detection method based on starting point guidance. BACKGROUND
[0002] Automatic driving technology has become a research hotspot in the academic and industrial circles. Lane line detection plays a crucial role in the automatic driving system, especially in the advanced driving assistance system (ADAS). At present, the lane line detection method based on deep learning can be divided into four categories: segmentation-based method; line anchor / row anchor-based method; parameter-based method; and key point-based method. The line anchor-based method is similar to target detection, which regresses the lane line by predicting the offset between the line anchor and the point on the lane line. However, the preset line anchor also makes this kind of method lack flexibility, and the quality of the line anchor is low. The strong shape prior makes it difficult for the model to describe lane lines of different shapes, especially when facing various complex lane line types, it is difficult to adapt. SUMMARY
[0003] In order to solve the above-mentioned defects in the prior art, the present application provides a lane line detection method based on starting point guidance, which provides additional lane line spatial feature information through a starting point detection head; according to the predicted starting point of the lane line, the line anchor is screened, which can reduce the number of line anchors and improve the quality of the line anchors, thereby improving the accuracy and robustness of lane line detection.
[0004] The object of the present application can be achieved by adopting the following technical solutions:
[0005] A lane line detection method based on starting point guidance, the lane line detection method comprising the following steps:
[0006] S1, collecting lane line image data sets of road scenes, and dividing the data sets into a training set and a validation set, and performing data enhancement on the lane line images;
[0007] S2, generating a true value data label file of the lane line image data set;
[0008] S3, constructing a lane line detection network, the lane line detection network adopting a line anchor-based lane line detection network as a framework, comprising a backbone network, a neck network and a detection head network connected in sequence; the detection head network comprises a starting point detection head and a lane line detection head connected in parallel; the lane line detection head comprises a classification branch and a regression branch connected in parallel;
[0009] S3.1, the backbone network extracts features of the input lane line image to obtain hierarchical features P0, P1, …, Pn of n different semantic levels; i , …, P (n-1)wherein P0 represents the highest semantic level of the hierarchical feature; the n-layer hierarchical feature is input into the neck network for feature fusion to obtain n-layer hierarchical fusion features F0, F1, …, Fn-1, and Fn. i , …, Fn-1 (n-1) wherein F0 represents the lowest resolution of the hierarchical fusion feature; the n-layer hierarchical fusion feature is input into the detection head network;
[0010] S3.2, the n-layer hierarchical fusion feature is input into the starting point detection head in the detection head network, the starting point detection head predicts the lane line starting point coordinates, and the NMS non-maximum suppression algorithm is used for post-processing to filter the redundant starting points to obtain the processed predicted lane line starting point coordinates;
[0011] S3.3, the lane line detection head in the detection head network takes the preset line anchor as the candidate lane line, the lane line detection head screens the preset line anchor according to the lane line starting point coordinates predicted in step S3.2; the n-layer hierarchical fusion feature is input into the lane line detection head in the detection head network, the anchor point position and the starting point position of the line anchor are regressed to the real lane line position by the regression branch in the lane line detection head, and the starting point angle of the line anchor is regressed to the real lane line starting point angle; the lane line detection head screens the line anchor output by the regression prediction again according to the lane line starting point coordinates predicted in step S3.2; the anchor point position of the screened line anchor is taken as the obtained lane line position; the classification branch in the lane line detection head outputs the corresponding lane line confidence;
[0012] S4, the lane line detection network is trained and verified using the training set and the verification set until convergence, and the model parameters of the lane line detection network are obtained;
[0013] S5, the lane line detection network loads the model parameters, inputs the lane line image of the road scene to be detected, and obtains the detection result of the lane line, wherein the detection result of the lane line includes the lane line position and the lane line confidence.
[0014] Further, in step S3.3, the lane line detection head calculates the distance between the predicted lane line starting point and the line anchor starting point using the lane line starting point coordinates predicted in step S3.2, screens the preset line anchor and the line anchor output by the regression prediction, and only retains the line anchor with a distance less than a preset threshold; the closer the starting point of the line anchor to the predicted lane line starting point, the higher the probability of corresponding to the real lane line, thereby improving the quality of the line anchor and reducing the number of line anchors that need to participate in subsequent operation through line anchor screening; different preset thresholds are adopted for different levels of hierarchical fusion features to adapt to the feature scales of different levels and control the number of retained line anchors of different levels respectively.
[0015] Furthermore, in step S3.3, the lane line starting point coordinates predicted in step S3.2 are used to enhance the n-layer hierarchical fusion features input to the lane line detection head, thereby further improving the performance of the lane line detection head. The process is as follows:
[0016] According to the hierarchical fusion characteristics F (n-1) Size and structural features heatmap M (n-1) A Gaussian kernel is used at the starting point of each predicted lane line. Expanded into a feature heatmap, where The X and Y coordinates of the predicted j-th starting point are given, the hyperparameter σ is the variance of the Gaussian kernel, and (x,y) are the X and Y coordinates of the heatmap. If there is overlap between the Gaussian kernels of two starting points, the maximum value is taken. Starting from the (n-1)-th layer, the feature heatmap M of the i-th layer is downsampled. i The size is adjusted layer by layer to the (i-1)th layer level fusion feature F (i-1) The size is used to obtain the feature heatmap M of the (i-1)th layer. (i-1) The n-layer feature heatmap M is obtained. (n-1) M i M1, M0; then the i-th layer feature heatmap M is concatenated using the concat operation. i The corresponding i-th layer fusion feature F i By concatenating the features separately, we obtain the enhanced hierarchical fusion feature E of the i-th layer. i The formula is expressed as E i =concat(M i ,F i The n-layer enhanced hierarchical fusion feature is input to the lane line detection head and outputs the lane line position and the corresponding lane line confidence. The enhanced hierarchical fusion feature integrates the starting point position information into the hierarchical fusion feature, which enhances the feature representation capability of the hierarchical fusion feature and further improves the performance of the lane line detection head.
[0017] Furthermore, in step S3.2, the process by which the starting point detection head predicts the coordinates of the starting point of the lane line is as follows:
[0018] The feature extraction network of the starting point detection head consists of multiple CBR convolutional modules, a convolutional Conv layer, and a Sigmoid activation function layer connected in sequence. Each CBR convolutional module comprises a convolutional Conv layer, a batch normalization (BN) layer, and a ReLU activation function layer connected in sequence. The CBR convolutional modules extract features, the BN layer improves gradient propagation, accelerates convergence, and enhances generalization ability, the ReLU activation function layer strengthens non-linear feature representation, and the Sigmoid activation function layer normalizes the output to the (0,1) interval, representing probability. The feature F is then fused hierarchically.(n-1) The feature extraction network of the input starting point detection head obtains a confidence heat map M, the confidence heat map M reflects the probability that each position of the heat map is identified as a lane line starting point, the probability of a position closer to the predicted starting point is closer to 1, and the probability of a position farther from the predicted starting point is closer to 0; the peak value of the detection confidence heat map M is detected to obtain the predicted lane line starting point coordinates, and then the NMS non-maximum suppression algorithm is used for post-processing to filter redundant starting points and improve the accuracy of starting point detection, thereby obtaining the processed predicted lane line starting point coordinates.
[0019] Further, in the step S3.3, a plurality of straight lines with different preset starting points and different preset horizontal angles are generated as preset line anchors, the preset horizontal angle is set as the starting point angle of the line anchor, and the line anchor is uniformly sampled at N A equidistant points in the vertical Y-axis direction as anchor points; the preset line anchor is used as the initial position of the candidate lane line, the preset line anchor is generated by using a plurality of different preset starting points and a plurality of different preset horizontal angles, and lane lines at different positions and different horizontal angles in the predicted image are predicted; the position of the line anchor is corrected, and the line anchor can approximate lane lines of different shapes.
[0020] Further, in the step S3.3, the lane line detection head uses the screened line anchor as the candidate lane line, samples the input hierarchical fusion features by using the anchor point position of the line anchor to form a feature vector, obtains the features of the lane line described by the line anchor, and inputs the feature vector into the regression branch and the classification branch in the lane line detection head; the regression branch performs regression prediction to regress the anchor point position and the starting point position of the line anchor to the real lane line position, and to regress the starting point angle of the line anchor to the real lane line starting point angle, wherein the real lane line starting point angle is the horizontal angle of the tangent line of the real lane line starting point; the regression branch outputs the offset of the anchor point position, the starting point position and the starting point angle of the line anchor, and corrects the anchor point position, the starting point position and the starting point angle of the line anchor; the classification branch outputs the lane line confidence, and the greater the confidence, the greater the probability that the line anchor corresponds to the real lane line.
[0021] Further, the regression branch and the classification branch are composed of a plurality of sequentially connected fully connected layer modules and one fully connected layer, wherein the fully connected layer module is composed of one fully connected layer and one ReLU activation function layer connected in sequence, and the ReLU activation function layer enhances the non-linear feature representation capability.
[0022] Further, in the step S3, the backbone network adopts one of ResNet, DLA, CSPDarkNet, MobileNet, ShuffleNet, DenseNet, VGG, RepVGG and EfficientNet network, and the backbone network is selected according to system performance requirements and calculation complexity requirements; the neck network adopts FPN or PAN network, and the neck network is selected according to system performance requirements and calculation complexity requirements.
[0023] The present application has the following beneficial effects relative to the prior art:
[0024] (1) The present application introduces a starting point detection head to capture more spatial feature information of lane lines, which helps to more accurately locate lane lines.
[0025] (2) The present application designs a line anchor screening method based on starting point guidance, which can reduce the number of line anchors while improving the quality of line anchors, thereby improving the accuracy and robustness of lane line detection.
[0026] (3) The present application designs a method of enhancing hierarchical fusion features using starting point coordinates to improve the performance of the lane line detection head. BRIEF DESCRIPTION OF DRAWINGS
[0027] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor based on the structures shown in the drawings. The following is explained:
[0028] Figure 1 is a flowchart of the lane line detection method based on starting point guidance disclosed by the present application;
[0029] Figure 2 is a structure schematic diagram of the lane line detection network model based on starting point guidance disclosed by the present application;
[0030] Figure 3 is a structure schematic diagram of the lane line detection network model based on starting point guidance disclosed by the present application;
[0031] Figure 4 is a structure schematic diagram of the starting point detection head feature extraction network of embodiment 1 of the present application;
[0032] Figure 5 is a lane line detection result schematic diagram of embodiment 1 of the present application;
[0033] Figure 6This is a schematic diagram of the lane line detection results in Embodiment 2 of the present invention. Detailed Implementation
[0034] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0035] Example 1
[0036] This embodiment discloses a lane line detection method based on starting point guidance. The process of this lane line detection method is as follows: Figure 1 As shown, the lane line detection method includes the following steps:
[0037] S1. Collect a dataset of lane line images of road scenes and divide the dataset into a training set and a validation set;
[0038] In this embodiment, the publicly available lane line datasets Tusimple and CULane are used, and the images and their corresponding label information are divided into training and validation sets in an 8:1 ratio. Data augmentation is then performed on the lane line images.
[0039] S2. Generate the ground truth label file for the lane line image dataset;
[0040] S3. Construct a lane line detection network;
[0041] Lane detection networks such as Figure 2 As shown, the lane detection network adopts the anchor-based lane detection network CLRNet as its framework, which includes a backbone network, a neck network, and a detection head network connected in sequence. The detection head network includes a starting point detection head and a lane detection head connected in parallel. The lane detection head includes a classification branch and a regression branch connected in parallel.
[0042] S3.1 Both the backbone network and the neck network adopt the CLRNet network structure. The backbone network uses a ResNet18 network, and the neck network uses a Feature Pyramid Network (FPN). The backbone network extracts features from the input lane line image, obtaining n layers of hierarchical features P0, P1, ..., P at different semantic levels. i , ..., P (n-1) Where P0 represents the hierarchical feature at the highest semantic level; n hierarchical features are input into the neck network for feature fusion to obtain n hierarchical fused features F0, F1, ... F i F (n-1)wherein F0 represents the lowest resolution level fused feature; inputting the n level fused features into the detection head network; in this embodiment, n = 3.
[0043] S3.2, inputting the n level fused features into the starting point detection head in the detection head network; the feature extraction network of the starting point detection head is composed of a CBR convolution module, a convolution layer with a 3x3 convolution kernel and a Sigmoid activation function layer connected in sequence, wherein the CBR convolution module is composed of a convolution layer with a 1x1 convolution kernel, a batch normalization layer and a ReLU activation function layer connected in sequence; the level fused feature F Figure 4 is input into the feature extraction network of the starting point detection head, the Sigmoid activation function layer normalizes the output to the interval (0, 1) to obtain a confidence heat map M, and the confidence heat map M reflects the probability of each position of the heat map being identified as a lane line starting point; the peak value of the detection confidence heat map M is detected to obtain the predicted lane line starting point coordinates, and the NMS non-maximum suppression algorithm is used for post-processing to filter redundant starting points to obtain the processed predicted lane line starting point coordinates. (n-1)
[0044] S3.3, the lane line detection head in the detection head network adopts the structure of the CLRNet network, including a classification branch and a regression branch connected in parallel; the classification branch and the regression branch are composed of two fully connected layer modules and a fully connected layer connected in sequence, wherein the fully connected layer module is composed of a fully connected layer and a ReLU activation function layer connected in sequence;
[0045] The lower half of the left boundary of the image is equally divided to obtain 12 end points, the lower half of the right boundary of the image is equally divided to obtain 12 end points, and the lower boundary of the image is equally divided to obtain 36 end points, which are used as preset starting points; the starting points of the left boundary are selected as horizontal angles of 28.8° and 57.6°, the starting points of the right boundary are selected as horizontal angles of 122.4° and 151.2°, and the starting points of the lower boundary are selected as horizontal angles of 36°, 72°, 108° and 144° as starting point angles; two straight lines with different preset horizontal angles are generated as preset line anchors respectively by the starting points of the left and right boundaries, and four straight lines with different preset horizontal angles are generated as preset line anchors respectively by the starting points of the lower boundary, to obtain 192 preset line anchors; the line anchors are equally and uniformly sampled in the vertical Y axis direction to obtain N A = 36 in this embodiment. A
[0046] The lane line detection head calculates the distance between the predicted lane line starting point and the preset line anchor starting point using the predicted lane line starting point coordinates in step S3.2, and filters the preset line anchor, only keeping the preset line anchor with a distance less than a preset threshold;
[0047] The lane line detection head in the n-layer hierarchical fusion feature input detection head network takes the filtered preset line anchor as a candidate lane line, samples the input hierarchical fusion feature using the anchor point position of the line anchor to form a feature vector, and inputs the feature vector into the regression branch and the classification branch in the lane line detection head; the regression branch performs regression prediction to regress the anchor point position and the starting point position of the line anchor to the real lane line position, and to regress the starting point angle of the line anchor to the real lane line starting point angle, wherein the real lane line starting point angle is the horizontal angle of the tangent of the real lane line starting point; the lane line detection head calculates the distance between the predicted lane line starting point and the starting point of the line anchor output by the regression prediction, and filters the line anchor output by the regression prediction again, only keeping the line anchor with a distance less than a preset threshold; the anchor point position of the filtered line anchor is output as the lane line position; and the classification branch outputs the lane line confidence;
[0048] Different preset thresholds are used for different levels of hierarchical fusion features; in this embodiment, the thresholds corresponding to the 0th, 1st and 2nd layers are 0.9, 1.79 and 3.59, respectively;
[0049] S4, training and verifying the lane line detection network using the training set and the verification set until convergence is achieved, to obtain the model parameters of the lane line detection network;
[0050] S5, the lane line detection network loads the model parameters, inputs an image of a road scene to be detected, and obtains a detection result of the lane line, as shown in Figure 5 Figure 5 is a schematic diagram of the lane line detection result in this embodiment, and the detection result of the lane line includes the lane line position and the lane line confidence.
[0051] Embodiment 2
[0052] Based on embodiment 1, this embodiment further discloses a lane line detection method based on starting point guidance, and the flow of the lane line detection method is as shown in Figure 1 The lane line detection method comprises the following steps:
[0053] S1, collecting lane line image data sets of road scenes, and dividing the data sets into a training set and a verification set, which can be specifically referred to step S1 in embodiment 1;
[0054] S2, generating a true value data label file of the lane line image data set;
[0055] S3, construct a lane line detection network;
[0056] The lane line detection network adopts a lane line detection network CLRNet based on line anchors as a framework, and includes a backbone network, a neck network and a detection head network connected in sequence. Figure 3 The detection head network includes a starting point detection head and a lane line detection head connected in parallel.
[0057] S3.1, the backbone network and the neck network both adopt the structure of the CLRNet network, the backbone network adopts a ResNet18 network, and the neck network adopts a feature pyramid network FPN. i The backbone network extracts features from the input lane line image to obtain n layers of hierarchical features P0, P1, …, P (n-1) , where P0 represents the hierarchical feature of the highest semantic level. i The n layers of hierarchical features are input into the neck network for feature fusion to obtain n layers of hierarchical fusion features F0, F1, … F (n-1) , where F0 represents the hierarchical fusion feature with the lowest resolution. (n-1) (n-1) In this embodiment, n = 3.
[0058] S3.2, the n layers of hierarchical fusion features are input into the starting point detection head in the detection head network, which predicts the lane line starting point coordinates.
[0059] S3.3, the n layers of hierarchical fusion features input into the lane line detection head are enhanced using the predicted lane line starting point coordinates to further improve the performance of the lane line detection head, and the process is as follows:
[0060] According to the size of the hierarchical fusion feature F (n-1) , a feature heat map M (n-1) is constructed, and a Gaussian kernel is used to expand the feature heat map at each predicted lane line starting point position, where are the X-axis and Y-axis coordinates of the jth predicted starting point, σ is the variance of the Gaussian kernel, and (x, y) are the X-axis and Y-axis coordinates of the heat map. If there is an overlap between the Gaussian kernels of two starting points, the maximum value is taken. Starting from the (n-1)th layer, the size of the i-th layer feature heat map M i is adjusted layer by layer to the size of the (i-1)th layer hierarchical fusion feature F (i-1) to obtain the (i-1)th layer feature heat map M (i-1) , and n layers of feature heat maps M (n-1) , …, M iM0; the i-th layer feature heat map M i corresponding i-th layer hierarchical fusion feature F i respectively, to obtain the i-th layer enhanced hierarchical fusion feature E i , which is expressed by the formula E i = concat(M i , F i ); the n-layer enhanced hierarchical fusion feature is input into the lane line detection head; in this embodiment, σ = 2;
[0061] The lane line detection head in the detection head network adopts the structure of the CLRNet network, including a classification branch and a regression branch connected in parallel; the classification branch and the regression branch are composed of two sequentially connected full connection layer modules and a full connection layer, wherein the full connection layer module is composed of a full connection layer and a ReLU activation function layer connected in sequence;
[0062] The lower half of the left boundary of the image is equally divided to obtain 12 endpoints, the lower half of the right boundary of the image is equally divided to obtain 12 endpoints, and the lower boundary of the image is equally divided to obtain 36 endpoints, which are used as preset starting points; the starting points of the left boundary are selected as horizontal angles of 28.8° and 57.6°, the starting points of the right boundary are selected as horizontal angles of 122.4° and 151.2°, and the starting points of the lower boundary are selected as horizontal angles of 36°, 72°, 108° and 144° as starting point angles; two straight lines with different preset horizontal angles are generated as preset line anchors using the starting points of the left and right boundaries respectively, and four straight lines with different preset horizontal angles are generated as preset line anchors using the starting points of the lower boundary respectively, to obtain 192 preset line anchors; the line anchors are equally and uniformly sampled in the vertical Y-axis direction to obtain N A points as anchor points; in this embodiment, N A = 36;
[0063] The lane line detection head uses the predicted lane line starting point coordinates in step S3.2 to calculate the distance between the predicted lane line starting point and the starting point of the preset line anchor, and filters the preset line anchor, only keeping the preset line anchor with a distance less than a preset threshold;
[0064] The lane line detection head in the n-layer enhanced hierarchical fusion feature input detection head network uses the screened preset line anchor as a candidate lane line, samples the input enhanced hierarchical fusion feature using the anchor point position of the line anchor, forms a feature vector, and inputs the feature vector into a regression branch and a classification branch in the lane line detection head; the regression branch performs regression prediction, and regresses the anchor point position and the starting point position of the line anchor to the real lane line position and regresses the starting point angle of the line anchor to the real lane line starting point angle, wherein the real lane line starting point angle is the horizontal included angle of the tangent of the real lane line starting point; the lane line detection head calculates the distance between the predicted lane line starting point and the line anchor starting point output by the regression prediction, and screens the line anchor output by the regression prediction again, and only keeps the line anchor with a distance less than a preset threshold; the anchor point position of the screened line anchor is output as the lane line position; and the classification branch outputs the lane line confidence.
[0065] Different preset thresholds are used for different levels of the enhanced hierarchical fusion feature; in this embodiment, the thresholds corresponding to the 0th, 1st and 2nd layers are 0.9, 1.79 and 3.59 respectively.
[0066] S4, training and verifying the lane line detection network using the training set and the verification set until convergence is achieved, to obtain model parameters of the lane line detection network;
[0067] S5, the lane line detection network loads the model parameters, inputs an image of a road scene to be detected, and obtains a detection result of a lane line, as shown in Figure 6 Figure 6 is a schematic diagram of the lane line detection result in this embodiment, and the detection result of the lane line includes a lane line position and a lane line confidence.
[0068] The above embodiment is a preferred embodiment of the present application, but the embodiments of the present application are not limited to the above embodiment, and any change, modification, replacement, combination or simplification without departing from the spirit and principle of the present application should be an equivalent replacement mode, and all are included in the protection scope of the present application.
Claims
1. A lane line detection method based on a starting point guidance, characterized by, The lane line detection method comprises the following steps: S1, collecting lane line image data sets of road scenes, and dividing the data sets into a training set and a verification set, and performing data enhancement on the lane line images; S2, generating a true value data label file of the lane line image data set; S3, constructing a lane line detection network, which adopts a lane line detection network based on line anchors as a framework, comprising a backbone network, a neck network and a detection head network connected in sequence; the detection head network comprises a starting point detection head and a lane line detection head connected in parallel; the lane line detection head comprises a classification branch and a regression branch connected in parallel; S3.1, the backbone network extracts features from the input lane line image to obtain n-level hierarchical features P0, P1, …, P i , …, P (n-1) , where P0 represents the highest semantic level hierarchical feature; the n-level hierarchical features are input into the neck network for feature fusion to obtain n-level hierarchical fusion features F0, F1, …, F i , …, F (n-1) , where F0 represents the lowest resolution hierarchical fusion feature; the n-level hierarchical fusion features are input into the detection head network; S3.2, inputting the n-layer level fusion features into the starting point detection head in the detection head network, the starting point detection head predicting lane line starting point coordinates, and performing post-processing by using an NMS non-maximum suppression algorithm to filter redundant starting points to obtain processed predicted lane line starting point coordinates; S3.3, the lane line detection head in the detection head network taking a preset line anchor as a candidate lane line, the lane line detection head screening the preset line anchor according to the predicted lane line starting point coordinates in step S3.2; the n-layer level fusion features are input into the lane line detection head in the detection head network, the regression branch in the lane line detection head performing regression prediction by using the screened preset line anchor, and the anchor point position and the starting point position of the line anchor being regressed to the real lane line position, and the starting point angle of the line anchor being regressed to the real lane line starting point angle; the lane line detection head screening the line anchor output by the regression prediction again according to the predicted lane line starting point coordinates in step S3.2; the anchor point position of the screened line anchor is taken as the obtained lane line position; the classification branch in the lane line detection head outputs the corresponding lane line confidence; S4, training and verifying the lane line detection network by using the training set and the verification set until convergence is achieved, and obtaining model parameters of the lane line detection network; S5, the lane line detection network loading the model parameters, inputting a lane line image of a road scene to be detected, and obtaining a lane line detection result, the lane line detection result comprising a lane line position and a lane line confidence.
2. The lane line detection method based on starting point guidance according to claim 1, wherein, In step S3.3, the lane line detection head calculates the distance between the predicted lane line starting point and the line anchor starting point by using the predicted lane line starting point coordinates in step S3.2, screens the preset line anchor and the line anchor output by the regression prediction, and only keeps the line anchor with a distance less than a preset threshold; different preset thresholds are adopted for different levels of the level fusion features.
3. The method of claim 1, wherein, In step S3.3, the n-layer level fusion features input into the lane line detection head are enhanced by using the predicted lane line starting point coordinates in step S3.2, and the process is as follows: According to the hierarchical fusion feature F (n-1) The size configuration feature heat map M (n-1) , a Gaussian kernel is used at each predicted lane line starting point position Extended to a feature heat map, wherein is the X-axis and Y-axis coordinates of the jth predicted starting point, the hyperparameter σ is the variance of the Gaussian kernel, (x, y) is the heat map X-axis and Y-axis coordinates; if there is overlap between the two starting point Gaussian kernels, take the maximum value; starting from the (n-1)th layer, the size of the ith layer feature heat map M i is adjusted to the size of the (i-1)th layer hierarchical fusion feature F (i-1) through downsampling operation, to obtain the (i-1)th layer feature heat map M (i-1) , to obtain n layer feature heat maps M (n-1) , …, M i , …, M1, M0; then the ith layer feature heat map M i is spliced with the corresponding ith layer hierarchical fusion feature F i respectively through concat splicing operation, to obtain the ith layer enhanced hierarchical fusion feature E i , the formula is E i = concat(M i , F i ); the n layer enhanced hierarchical fusion feature is input into the lane line detection head, and the lane line position and the corresponding lane line confidence are output.
4. The method of claim 1, wherein, In step S3.2, the process of predicting the lane line starting point coordinates by the starting point detection head is as follows: The feature extraction network of the starting point detection head comprises a plurality of CBR convolution modules, a convolution Conv layer and a Sigmoid activation function layer connected in sequence, wherein the CBR convolution module comprises a convolution Conv layer, a batch normalization BN layer and a ReLU activation function layer connected in sequence; Fusion feature F (n-1) The feature extraction network of the input starting point detection head obtains a confidence heat map M, and the confidence heat map M reflects the probability of each position of the heat map being identified as a starting point of a lane line; a peak value of the confidence heat map M is detected to obtain a predicted lane line starting point coordinate, and an NMS non-maximum suppression algorithm is used for post-processing to filter redundant starting points, thereby obtaining a processed predicted lane line starting point coordinate.
5. The method of claim 1, wherein, In the step S3.3, a plurality of straight lines with different preset horizontal angles are generated as preset line anchors with different preset starting points, the preset horizontal angle is set as the starting point angle of the line anchor, and the line anchor is evenly sampled N points in the vertical Y-axis direction as anchor points. A In the step S3.3, a plurality of straight lines with different preset horizontal angles are generated as preset line anchors with different preset starting points, the preset horizontal angle is set as the starting point angle of the line anchor, and the line anchor is evenly sampled N points in the vertical Y-axis direction as anchor points.
6. The method of claim 1, wherein, In the step S3.3, the lane line detection head takes the screened line anchor as a candidate lane line, samples the input hierarchical fusion feature using the anchor point position of the line anchor to form a feature vector, and inputs the feature vector into the regression branch and the classification branch in the lane line detection head; the regression branch performs regression prediction to regress the anchor point position and the starting point position of the line anchor to the real lane line position, and to regress the starting point angle of the line anchor to the real lane line starting point angle, wherein the real lane line starting point angle is the horizontal included angle of the tangent of the real lane line starting point; The classification branch outputs the lane line confidence.
7. The method of claim 6, wherein, The regression branch and the classification branch are composed of a plurality of fully connected layer modules connected in sequence and a fully connected layer, wherein the fully connected layer module is composed of a fully connected layer and a ReLU activation function layer connected in sequence.
8. The method of claim 1, wherein, In the step S3, the backbone network adopts one of ResNet, DLA, CSPDarkNet, MobileNet, ShuffleNet, DenseNet, VGG, RepVGG and EfficientNet network; and the neck network adopts FPN or PAN network.