A lane line detection method

Abstract

The present invention provides a lane line detection method, comprising the following steps: S1: acquiring a road image and preprocessing the road image to obtain an input image; S2: using a lane line recognition model to identify whether lane lines exist in the input image; if so, executing step S3; otherwise, executing step S5; S3: using the lane line detection model to detect the input image, mapping the detection results to the coordinates of the lane lines in the input image, and obtaining lane line coordinate points; S4: selecting lane line coordinate points in the middle portion of the lane line for fitting processing, and obtaining the optimal fitting lane line as the output result; S5: terminating the process. The present invention provides a lane line detection method that addresses the problem that current lane line detection technology is still imperfect.

Description

Technical Field

[0001] The present invention relates to the technical field of lane line detection, and more specifically, to a lane line detection method. Background Art

[0002] Lane detection is generally defined as detecting the location of lane lines in a video or image of a vehicle moving along the road. Lane detection is a crucial component of both assisted and autonomous driving systems. Lane departure warnings and lane keeping assist can promptly correct driver errors, reducing traffic accidents caused by incorrect operation and fatigue, thereby effectively ensuring driving safety and lowering the accident rate.

[0003] In recent years, the foundations of autonomous driving have matured and become increasingly applicable. Lane detection is considered a key issue in the field. This task has three key aspects: first, detecting the specific location of lane lines at a higher semantic level, even when they are partially or completely obscured; second, detecting the direction of lane lines based on current road conditions. For example, at traffic lights or on sidewalks, it is necessary to identify and determine whether lane lines are present ahead. Third, speed is crucial. Lane detection algorithms in autonomous driving must ensure accuracy while meeting real-world requirements.

[0004] Thanks to the powerful feature extraction capabilities of deep learning models, existing lane detection technology primarily relies on deep learning methods. Lane detection technologies based on deep learning methods can generally be divided into two categories: semantic segmentation-based methods and line classification-based methods. Semantic segmentation-based methods offer high detection accuracy, but are limited by complex encoding and decoding networks and the need to classify all image pixels. Consequently, they are slow and unsuitable for edge detection devices. Line classification-based methods offer fast detection speeds and real-time performance, but can suffer from overfitting due to the similarity of lane structures, resulting in reduced detection accuracy. Therefore, current lane detection technology is far from perfect and cannot simultaneously meet the requirements for both speed and accuracy in practical applications. Summary of the Invention

[0005] The present invention provides a lane line detection method to overcome the technical defect that the current lane line detection technology is not yet perfect.

[0006] In order to solve the above technical problems, the technical solutions of the present invention are as follows:

[0007] A lane line detection method includes the following steps:

[0008] S1: Acquire a road image and preprocess the road image to obtain an input image;

[0009] S2: Use the lane line recognition model to identify whether there is a lane line in the input image.

[0010] If there is a lane line, execute step S3.

[0011] If there is no lane line, the output result is no lane line, and step S5 is executed;

[0012] The lane line recognition model is built based on the convolutional neural network Vgg16;

[0013] S3: Use the lane line detection model to detect the input image, map the detection results to the coordinates of the lane line in the input image, and obtain the lane line coordinate points;

[0014] The lane detection model is built based on the ResNeSt18 feature extraction network of the improved segmentation attention model;

[0015] S4: Select the lane line coordinate points in the middle part of the lane line for fitting processing, and obtain the optimal fitting lane line as the output result;

[0016] S5: Execution ends.

[0017] In the above scheme, the lane line recognition model is first used to screen the road image to prevent road images without lane lines from entering the detection stage, thereby reducing meaningless misidentification in subsequent detections; then, the ResNeSt18 feature extraction network lane line detection model based on the improved segmentation attention model is used to detect the input image, ensuring a faster detection speed, deepening the number of network layers, improving the network learning ability, realizing grid-level mask detection of lane lines, and improving detection accuracy; finally, the lane line coordinate points in the middle part of the lane line are selected for fitting processing, which reduces the misidentification rate and further improves the fitting accuracy.

[0018] Preferably, the preprocessing in step S1 includes image grayscale processing and image filtering processing.

[0019] In the above scheme, the amount of data is reduced by graying the image, thereby improving the real-time performance of the detection algorithm; and the interference of uncertain noise points in the road image is reduced by image filtering.

[0020] Preferably, the improvement of the segmentation attention model of the ResNeSt18 feature extraction network is specifically as follows: the feature map of the input segmentation attention model is first convolved, activated and pooled, and then the feature map is divided into multiple sub-feature maps, and then each sub-feature map is allowed to enter a different sub-channel respectively, and the features of each sub-channel are obtained through 1*1 convolutional layer learning to form an attention mechanism, and finally the features of each sub-channel are fused and the fused feature map is output.

[0021] In this solution, only one 1x1 convolutional layer is used to learn the features of each sub-channel, discarding the original global pooling and two fully connected layers that learn per-channel features. Because the 1x1 convolutional layer is only used to learn feature coefficients and does not directly involve the feature map that requires forward propagation, it has a minimal performance loss while reducing complexity and improving inference speed, making it highly competitive in embedded devices.

[0022] Preferably, the lane detection model is combined with the semantic segmentation network U-Net for multi-task learning during training, and weights are updated by back propagation;

[0023] The lane detection model consists of four layers, and the feature maps output by the first three layers serve as the input of the semantic segmentation network U-Net.

[0024] In the above solution, during the training phase, the lane detection model and the semantic segmentation network U-Net are combined to form multi-task learning. The weights of the lane detection model are better optimized through backpropagation, giving the lane detection model global awareness and expanding the receptive field of the lane detection model.

[0025] Preferably, before inputting the input image into the lane line detection model for detection, the following steps are further included:

[0026] A1: compress the input image to a preset size;

[0027] A2: Perform data augmentation on the input image, including random image rotation and left-right shifting.

[0028] A3: The input image is processed in two steps in parallel, serving as the input to the lane detection model and the label for the semantic segmentation network U-Net. The specific processing steps are as follows:

[0029] A3.1: Divide the input image into w x h grids and (combined with the lane recognition model's results) label each grid as to whether a lane line exists. w represents the number of columns into which the input image is divided, and h represents the number of rows into which the input image is divided.

[0030] A3.2: Compress the input image into label data suitable for the semantic segmentation network U-Net network structure.

[0031] In the above scheme, the size of the input image is compressed to improve the detection speed, and the input image is subjected to data enhancement processing to prevent overfitting.

[0032] Preferably, the overall loss function after combining the lane detection model and the semantic segmentation network U-Net is defined as:

[0033] loss=α*L cls +β*L sim +γ*L shp +μ*L seg

[0034] Among them, L cls is the classification loss function of the lane line detection model, L sim is the similarity loss function for constraining the lane line structure, L shap is the shape loss function that constrains the lane line structure, L seg is the cross entropy loss function of the semantic segmentation network U-Net, α is L cls The weight value of β is L sim The weight value of γ is L shp The weight value of μ is L seg The weight value of .

[0035] Preferably,

[0036] The classification loss function of the lane detection model is defined as:

[0037] L cls =NllLoss[(1-softmax(P logit )) gamma *log(softmax(P logit )),Label]ΛΛ (1)

[0038] Among them, the NllLoss function represents the loss value of the corresponding position according to the label, P logit represents the logit probability value output by the lane detection model, and softmax(.) represents the softmax normalization operation of the lane detection model output probability;

[0039] The similarity loss function of the constraint lane line structure is defined as:

[0040]

[0041] Among them, P i,j represents the probability value of the existence of lane lines in the i-th lane and the j-th row, C represents the total number of lanes, and h represents the number of rows into which the input image is divided;

[0042] The shape loss function of the constraint lane line structure is defined as:

[0043]

[0044] Among them, Loc i,j Indicates the position information of the jth lane line,

[0045]

[0046] w represents the number of columns into which the input image is divided, k represents the kth column of the input image, and K represents the number of columns of the training set used during training;

[0047] The cross entropy loss function of the semantic segmentation network U-Net is defined as:

[0048]

[0049] Where M represents the number of categories; y ic represents the sign function, which takes 1 if the true category of sample i is equal to c, otherwise it takes 0; p ic It represents the predicted probability that the observed sample i belongs to category c.

[0050] In the above scheme, a similarity loss function is defined to constrain the lane line structure, so that the output lane line is more likely to be a continuous line segment, and the L2 norm is used to increase the constraints on the lane line shape. A shape loss function is defined to constrain the lane line structure, and the second-order difference of position information is used to make the obtained lane line more likely to be a straight line.

[0051] The overall loss function is composed of the classification loss function of the lane detection model, the similarity loss function of the constrained lane line structure, the shape loss function of the constrained lane line structure, and the cross-entropy loss function of the semantic segmentation network U-Net, combined with corresponding weights. When the loss values ​​of these four loss functions all reach the ideal value, the overall loss function converges and the lane detection model training is completed; otherwise, the lane detection model and the semantic segmentation network U-Net are forced to adjust their weights through deep learning backpropagation until the overall loss function converges and achieves the global optimal solution.

[0052] Preferably, in step S3, the horizontal coordinate value X of the jth lane line of the i-th lane is calculated by the following formula: ij and the vertical coordinate value Y ij , thereby mapping the detection results to the coordinates of the lane lines in the input image,

[0053] X ij The calculation formula is:

[0054]

[0055] Y coordinate value ij The calculation formula is:

[0056] Y ij =anchor[j-1-k]

[0057] Among them, W _col , W _img, w _input They represent the image division grid width, the original image width, and the image width of the input lane line detection model respectively. Anshor represents the initial vertical coordinate set of the division grid, k∈(0, j).

[0058] In the above scheme, the position of each lane line in the w column of the input image is first calculated using formula (4), and then the corresponding horizontal and vertical coordinate values ​​are calculated to obtain the coordinates of the lane line in the input image.

[0059] Preferably, in step S4, the lane line coordinate points in the middle part of the lane line are selected for fitting processing, specifically: 1 / 5 of the lane line coordinate points at both ends of the lane line are removed, and the retained lane line coordinate points are subjected to least squares fitting processing.

[0060] In the above scheme, since the middle part of the detected lane line coordinate points is relatively stable and the misrecognition rate is low, in order to improve the fitting accuracy, 3 / 5 of the lane line coordinate points in the middle part of the lane line are selected for fitting processing. That is, after removing 1 / 5 of the lane line coordinate points at the beginning of the lane line and 1 / 5 of the lane line coordinate points at the end of the lane line, the least squares method is used to fit the remaining lane line coordinate points.

[0061] Preferably, after obtaining the optimal fitting lane line in step S4, the method further includes realizing vehicle lane change recognition in combination with the optimal fitting lane line, specifically: presetting a vehicle lane change slope threshold, and detecting the slope of the output optimal fitting lane line in real time,

[0062] If the slope of the optimal fitting lane line does not exceed the preset vehicle lane change slope threshold, it is recognized that the vehicle has not changed lanes.

[0063] If the slope of the best-fit lane line exceeds the preset vehicle lane change slope threshold, it is recognized that the vehicle has changed lanes.

[0064] In the above scheme, the slope change of the optimal fitting lane line is used to determine whether the vehicle has changed lanes, which can accurately identify the vehicle's driving status with fast recognition speed and high stability.

[0065] Compared with the prior art, the beneficial effects of the technical solution of the present invention are:

[0066] The present invention provides a lane line detection method. First, a lane line recognition model is used to screen road images to prevent road images without lane lines from entering the detection stage, thereby reducing meaningless misrecognition in subsequent detections. Then, a lane line detection model based on the ResNeSt18 feature extraction network of an improved segmentation attention model is used to detect the input image, thereby ensuring a faster detection speed, increasing the number of network layers, improving the network learning ability, realizing grid-level mask detection of lane lines, and improving detection accuracy. Finally, the lane line coordinate points in the middle part of the lane line are selected for fitting processing, thereby reducing the misrecognition rate and further improving the fitting accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] Figure 1 It is a flowchart of the steps for implementing the technical solution of the present invention;

[0068] Figure 2 Schematic diagram of the overall structure of the ResNeSt18 feature extraction network based on the improved segmentation attention model in the present invention;

[0069] Figure 3 Schematic diagram of the structure of the improved segmentation attention model described in the present invention;

[0070] Figure 4 Schematic diagram of the data flow of the feature maps output by the first three layers of the lane detection model in the present invention into the semantic segmentation network U-Net;

[0071] Figure 5 Schematic diagram of the slope change trend of the best-fit lane lines on the left and right sides of the vehicle detected when the vehicle is driving normally in the present invention;

[0072] Figure 6 Schematic diagram of the slope change trend of the best-fit lane lines on the left and right sides of the vehicle detected when the vehicle changes lanes in the present invention;

[0073] Figure 7 This is a schematic diagram of the application of the present invention in a tunnel road condition;

[0074] Figure 8 This is a schematic diagram of the application of the present invention in a curved road condition;

[0075] Figure 9 This is a schematic diagram of the application of the present invention in a rural road situation;

[0076] Figure 10 This is a schematic diagram of the application of the present invention in urban road conditions;

[0077] Figure 11 This is a schematic diagram of the application of the present invention in a zebra crossing situation;

[0078] Figure 12 A schematic diagram of the application of the present invention in a scenario of road marking conditions;

[0079] Figure 13 This is a schematic diagram of the application of the present invention in a scenario with changing road conditions;

[0080] Figure 14 This is a schematic diagram of the application of the present invention in a scenario of road defects;

[0081] Figure 15 This is a schematic diagram of the application of the present invention in nighttime traffic conditions. DETAILED DESCRIPTION

[0082] The accompanying drawings are for illustrative purposes only and are not to be construed as limiting this patent;

[0083] In order to better illustrate this embodiment, some parts in the drawings may be omitted, enlarged, or reduced, and do not represent the actual product size;

[0084] It is understandable to those skilled in the art that some well-known structures and descriptions thereof may be omitted in the drawings.

[0085] The technical solution of the present invention is further described below with reference to the accompanying drawings and embodiments.

[0086] Example 1

[0087] like Figure 1 As shown, a lane line detection method includes the following steps:

[0088] S1: Acquire a road image and preprocess the road image to obtain an input image;

[0089] S2: Use the lane line recognition model to identify whether there is a lane line in the input image.

[0090] If there is a lane line, execute step S3.

[0091] If there is no lane line, the output result is no lane line, and step S5 is executed;

[0092] The lane line recognition model is built based on the convolutional neural network Vgg16;

[0093] S3: Use the lane line detection model to detect the input image, map the detection results to the coordinates of the lane line in the input image, and obtain the lane line coordinate points;

[0094] The lane detection model is built based on the ResNeSt18 feature extraction network of the improved segmentation attention model;

[0095] S4: Select the lane line coordinate points in the middle part of the lane line for fitting processing, and obtain the optimal fitting lane line as the output result;

[0096] S5: Execution ends.

[0097] During the specific implementation process, the lane line recognition model is first used to screen road images to prevent road images without lane lines from entering the detection stage, thereby reducing meaningless misrecognition in subsequent detections; then, the input image is detected using the ResNeSt18 feature extraction network lane line detection model based on the improved segmentation attention model, which ensures a faster detection speed, deepens the number of network layers, improves the network learning ability, realizes grid-level mask detection of lane lines, and improves detection accuracy; finally, the lane line coordinate points in the middle part of the lane line are selected for fitting processing to reduce the misrecognition rate and further improve the fitting accuracy.

[0098] Example 2

[0099] like Figure 1-4 As shown, a lane line detection method includes the following steps:

[0100] S1: Acquire a road image and preprocess the road image to obtain an input image;

[0101] More specifically, the preprocessing in step S1 includes image grayscale processing and image filtering processing.

[0102] In the specific implementation process, the image grayscale processing is used to reduce the amount of data and improve the real-time performance of the detection algorithm; image filtering processing is used to reduce the interference of uncertain noise points in road images.

[0103] S2: Use the lane line recognition model to identify whether there is a lane line in the input image.

[0104] If there is a lane line, execute step S3.

[0105] If there is no lane line, the output result is no lane line, and step S5 is executed;

[0106] The lane line recognition model is built based on the convolutional neural network Vgg16;

[0107] S3: Use the lane line detection model to detect the input image, map the detection results to the coordinates of the lane line in the input image, and obtain the lane line coordinate points;

[0108] The lane detection model is built based on the ResNeSt18 feature extraction network of the improved segmentation attention model;

[0109] More specifically, the improvement of the segmentation attention model of the ResNeSt18 feature extraction network is as follows: the feature map of the input segmentation attention model is first convolved, activated and pooled, and then the feature map is divided into multiple sub-feature maps, and then each sub-feature map is allowed to enter a different sub-channel respectively. The features of each sub-channel are obtained through 1*1 convolutional layer learning to form an attention mechanism, and finally the features of each sub-channel are fused to output the fused feature map.

[0110] In its implementation, it utilizes only one 1x1 convolutional layer to learn the features of each subchannel, discarding the traditional global pooling and two fully connected layers required to learn per-channel features. Because the 1x1 convolutional layer is used only to learn feature coefficients and does not directly involve feature maps that require forward propagation, it minimizes performance loss while reducing complexity and increasing inference speed, making it highly competitive in embedded devices.

[0111] More specifically, the lane detection model is trained with a semantic segmentation network U-Net for multi-task learning and weight updates are performed through back-propagation.

[0112] The lane detection model consists of four layers, and the feature maps output by the first three layers serve as the input of the semantic segmentation network U-Net.

[0113] During the specific implementation process, in the training phase, the lane detection model and the semantic segmentation network U-Net are combined to form multi-task learning. The weights of the lane detection model are better optimized through backpropagation, so that the lane detection model has global awareness and expands the receptive field of the lane detection model. Figure 4 The numbers in represent the number of channels of each layer of feature maps in the semantic segmentation network U-Net.

[0114] More specifically, before inputting the input image into the lane line detection model for detection, the following steps are also included:

[0115] A1: compress the input image to a preset size;

[0116] A2: Perform data augmentation on the input image, including random image rotation and left-right shifting.

[0117] A3: The input image is processed in two steps in parallel, serving as the input to the lane detection model and the label for the semantic segmentation network U-Net. The specific processing steps are as follows:

[0118] A3.1: Divide the input image into w x h grids and (combined with the lane recognition model's results) label each grid as to whether a lane line exists. w represents the number of columns into which the input image is divided, and h represents the number of rows into which the input image is divided.

[0119] A3.2: Compress the input image into label data suitable for the semantic segmentation network U-Net network structure.

[0120] In the specific implementation process, the size of the input image is compressed to improve the detection speed, and the input image is enhanced to prevent overfitting.

[0121] More specifically, the overall loss function after combining the lane detection model and the semantic segmentation network U-Net is defined as:

[0122] loss=α*L cls +β*L sim +γ*L shp +μ*L seg

[0123] Among them, L cls is the classification loss function of the lane line detection model, L sim is the similarity loss function for constraining the lane line structure, L shap is the shape loss function that constrains the lane line structure, L seg is the cross entropy loss function of the semantic segmentation network U-Net, α is L cls The weight value of β is L sim The weight value of γ is L shp The weight value of μ is L seg The weight value of .

[0124] More specifically,

[0125] The classification loss function of the lane detection model is defined as:

[0126] L cls =NllLoss[(1-softmax(P logit )) gamma *log(softmax(P logit )),Label]ΛΛ (1)

[0127] Among them, the NllLoss function represents the loss value of the corresponding position according to the label, P logit represents the logit probability value output by the lane detection model, and softmax(.) represents the softmax normalization operation of the lane detection model output probability;

[0128] The similarity loss function of the constraint lane line structure is defined as:

[0129]

[0130] Among them, P i,j represents the probability value of the existence of lane lines in the i-th lane and the j-th row, C represents the total number of lanes, and h represents the number of rows into which the input image is divided;

[0131] The shape loss function of the constraint lane line structure is defined as:

[0132]

[0133] Among them, Loc i,j Indicates the position information of the jth row of the i-th lane line. In this embodiment, only the first 1 / 8 of the lane lines are subject to shape constraints. In actual implementation, the shape constraint range can be adjusted accordingly according to actual conditions.

[0134]

[0135] w represents the number of columns into which the input image is divided, k represents the kth column of the input image, and K represents the number of columns of the training set used during training;

[0136] The cross entropy loss function of the semantic segmentation network U-Net is defined as:

[0137]

[0138] Where M represents the number of categories; y ic represents the sign function, which takes 1 if the true category of sample i is equal to c, otherwise it takes 0; p ic It represents the predicted probability that the observed sample i belongs to category c.

[0139] During the implementation, a similarity loss function is defined to constrain the lane line structure, so that the output lane line is more likely to be a continuous line segment, and the L2 norm is used to increase the constraints on the lane line shape. A shape loss function is also defined to constrain the lane line structure. Through the second-order difference of position information, the obtained lane line is made more likely to be a straight line.

[0140] The overall loss function is composed of the classification loss function of the lane line detection model, the similarity loss function of the constrained lane line structure, the shape loss function of the constrained lane line structure, and the cross entropy loss function of the semantic segmentation network U-Net, combined with corresponding weights. In this embodiment, a=0.35, b=0.3, c=0.05, and d=0.35 are set respectively. When the loss values ​​of these four loss functions all reach the ideal values, the overall loss function converges and the lane line detection model training is completed; otherwise, the lane line detection model and the semantic segmentation network U-Net are forced to adjust their weights through deep learning back propagation until the overall loss function converges to achieve the global optimal solution.

[0141] More specifically, in step S3, the horizontal coordinate value X of the jth lane line of the i-th lane is calculated by the following formula: ij and the vertical coordinate value Yi ij , thereby mapping the detection results to the coordinates of the lane lines in the input image,

[0142] X ij The calculation formula is:

[0143]

[0144] Y coordinate value ij The calculation formula is:

[0145] Yi ij =anchor[j-1-k]

[0146] Among them, W _col , W _img , w _input They represent the image division grid width, the original image width, and the image width of the input lane line detection model respectively. Anshor represents the initial vertical coordinate set of the division grid, k∈(0, j).

[0147] In the specific implementation process, the position of each lane line in the w column of the input image is first calculated using formula (4), and then the corresponding horizontal and vertical coordinate values ​​are calculated to obtain the coordinates of the lane line in the input image.

[0148] S4: Select the lane line coordinate points in the middle part of the lane line for fitting processing, and obtain the optimal fitting lane line as the output result;

[0149] More specifically, in step S4, the lane line coordinate points in the middle part of the lane line are selected for fitting processing as follows: 1 / 5 of the lane line coordinate points at both ends of the lane line are removed, and the retained lane line coordinate points are fitted using the least squares method.

[0150] During the specific implementation process, since the middle part of the detected lane line coordinate points is relatively stable and the misrecognition rate is low, in order to improve the fitting accuracy, 3 / 5 of the lane line coordinate points in the middle part of the lane line are selected for fitting processing. That is, after removing 1 / 5 of the lane line coordinate points at the beginning of the lane line and 1 / 5 of the lane line coordinate points at the end of the lane line, the least squares method is used to fit the remaining lane line coordinate points.

[0151] S5: Execution ends.

[0152] Example 3

[0153] This embodiment proposes a vehicle lane change recognition method based on a lane line detection method, including the following steps:

[0154] S1: Acquire a road image and preprocess the road image to obtain an input image;

[0155] S2: Use the lane line recognition model to identify whether there is a lane line in the input image.

[0156] If there is a lane line, execute step S3.

[0157] If there is no lane line, the output result is no lane line, and step S5 is executed;

[0158] The lane line recognition model is built based on the convolutional neural network Vgg16;

[0159] S3: Use the lane line detection model to detect the input image, map the detection results to the coordinates of the lane line in the input image, and obtain the lane line coordinate points;

[0160] The lane detection model is built based on the ResNeSt18 feature extraction network of the improved segmentation attention model;

[0161] S4: Select the lane line coordinate points in the middle part of the lane line for fitting processing, and obtain the optimal fitting lane line as the output result;

[0162] More specifically, after obtaining the optimal fitting lane line in step S4, the method further includes realizing vehicle lane change recognition in combination with the optimal fitting lane line, specifically: presetting a vehicle lane change slope threshold, and detecting the slope of the output optimal fitting lane line in real time,

[0163] If the slope of the optimal fitting lane line does not exceed the preset vehicle lane change slope threshold, it is recognized that the vehicle has not changed lanes.

[0164] If the slope of the best-fit lane line exceeds the preset vehicle lane change slope threshold, it is recognized that the vehicle has changed lanes.

[0165] In the specific implementation process, when the vehicle is driving normally, the slope change trend of the optimal fitting lane lines on the left and right sides of the vehicle is as follows: Figure 5 As shown in the figure, the slope change trend of the optimal fit lane line is relatively stable, and the slope value is between (-1,1). When the vehicle changes lanes, the slope change trend of the optimal fit lane lines on the left and right sides of the vehicle is as follows: Figure 6 As shown in the figure, when the vehicle changes lanes, the slope of the optimal fit lane line changes suddenly. After the vehicle completes the lane change, the slope of the optimal fit lane line tends to be stable compared to the lane change curve. Figure 5 and Figure 6 The researchers found that the slope of the optimally fitted lane line when a vehicle changes lanes is much higher than when the vehicle is driving normally. Therefore, using the slope change of the optimally fitted lane line to determine whether a vehicle has changed lanes can accurately identify the vehicle's driving status with high speed and stability.

[0166] S5: Execution ends.

[0167] Example 4

[0168] This embodiment uses a self-collected vehicle ADAS dataset, which includes 6,000 training images and 500 test images. Among them, 1,200 training images are used for convolutional neural network Vgg16 training, including both road images with lane lines and road images without lane lines. The remaining 4,800 training images are road images with lane lines, which are used for lane line detection model training. Taking into account the subsequent lane change recognition, this embodiment only detects the lane lines on the left and right sides closest to the vehicle. Training can be stopped when the overall loss function converges. Both the lane line recognition model and the lane line detection model are trained using the Titan RTX 3090 graphics card. The 500 test images include 9 scenarios: tunnel road conditions, curved road conditions, rural road conditions, urban road conditions, zebra crossing road conditions, road marking road conditions, road changing conditions, road defect road conditions and night road conditions. Figure 7-15 These are actual measured examples of the lane line detection method in 9 scenarios.

[0169] This embodiment uses the slope change of the optimal fit lane line obtained by fitting to perform lane change recognition. By collecting 100 ADAS videos as a test, a total of 132 vehicle lane changes were included, of which 45 lane changes occurred on nighttime roads. This embodiment uses test images to test in seconds, and counts the number of seconds the vehicle is in lane change and not in lane change to evaluate the performance of the described vehicle lane change recognition method in lane change recognition. The lane change recognition effect results of the described vehicle lane change recognition method in the test are shown in Table 1.

[0170] Table 1

[0171] Road conditions Accuracy (%) Specificity (%) Sensitivity (%) F1(％) daytime 96.1 96.4 94.5 95.4 at night 94.9 95.1 94.3 94.7

[0172] According to actual measurements, this method is also applicable to curved roads, damaged roads, etc., with good stability and strong universality.

[0173] Obviously, the above embodiments of the present invention are merely examples for the purpose of clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. Those skilled in the art will appreciate that other variations or modifications can be made based on the above description. It is not necessary and impossible to enumerate all embodiments here. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the claims of the present invention.

Claims

1. A lane line detection method, characterized in that: The following steps are involved: S1: Acquire a road image and preprocess the road image to obtain an input image; S2: Use the lane line recognition model to identify whether there is a lane line in the input image. If there is a lane line, execute step S3. If there is no lane line, the output result is no lane line, and step S5 is executed; The lane line recognition model is built based on the convolutional neural network Vgg16; S3: Use the lane line detection model to detect the input image, map the detection results to the coordinates of the lane line in the input image, and obtain the lane line coordinate points; The lane detection model is built based on the ResNeSt18 feature extraction network of the improved segmentation attention model; The improvement of the segmentation attention model of the ResNeSt18 feature extraction network is as follows: the feature map of the input segmentation attention model is first convolved, activated and pooled, and then the feature map is divided into multiple sub-feature maps. Each sub-feature map is then allowed to enter a different sub-channel. The features of each sub-channel are learned through a 1*1 convolutional layer to form an attention mechanism. Finally, the features of each sub-channel are fused and the fused feature map is output; S4: Select the lane line coordinate points in the middle part of the lane line for fitting processing, and obtain the optimal fitting lane line as the output result; S5: Execution ends.

2. A lane line detection method according to claim 1, characterized in that: The preprocessing in step S1 includes image grayscale processing and image filtering processing.

3. The lane line detection method according to claim 1, characterized in that: The lane detection model is combined with the semantic segmentation network U-Net for multi-task learning during training, and weights are updated through backpropagation. The lane detection model consists of four layers, and the feature maps output by the first three layers serve as the input of the semantic segmentation network U-Net.

4. The lane line detection method according to claim 3, characterized in that: Before the input image is fed into the lane detection model for detection, the following steps are also included: A1: compress the input image to a preset size; A2: Perform data augmentation on the input image, including random image rotation and left-right shifting. A3: The input image is processed in two steps in parallel, serving as the input to the lane detection model and the label for the semantic segmentation network U-Net. The specific processing steps are as follows: A3.1: Divide the input image into w * h grids and mark whether each grid has a lane line, where w Indicates the number of columns the input image is divided into, h Indicates the number of rows the input image is divided into; A3.2: Compress the input image into label data suitable for the semantic segmentation network U-Net network structure.

5. The lane line detection method according to claim 4, characterized in that: The overall loss function after combining the lane detection model and the semantic segmentation network U-Net is defined as: in, is the classification loss function of the lane detection model, is the similarity loss function for constraining the lane line structure, is the shape loss function that constrains the lane line structure, is the cross entropy loss function of the semantic segmentation network U-Net, for The weight value of for The weight value of for The weight value of for The weight value of .

6. A lane line detection method according to claim 5, characterized in that: The classification loss function of the lane detection model is defined as: in, The function represents the loss value of the corresponding position according to the label. Represents the output of the lane detection model Probability value, Indicates the output probability of the lane detection model Normalization operation; The similarity loss function of the constraint lane line structure is defined as: in, Indicates the Lane, No. The probability value of the line existing in the lane line, C represents the total number of lanes, h Indicates the number of rows the input image is divided into; The shape loss function of the constraint lane line structure is defined as: in, Indicates the Lane lines Row location information, w Indicates the number of columns the input image is divided into, k The first k List, K Represents the number of columns of the training set used during training; The cross entropy loss function of the semantic segmentation network U-Net is defined as: in, Indicates the number of categories; Represents a sign function, if the sample The true category is equal to Take 1, otherwise take 0; Represents an observation sample Belong to category The predicted probability of .

7. The lane line detection method according to claim 6, characterized in that: In step S3, the first Lane No. The horizontal coordinate value of the lane line and the vertical coordinate value , thereby mapping the detection results to the coordinates of the lane lines in the input image, Horizontal axis value The calculation formula is: Vertical axis value The calculation formula is: in, , , Respectively represent the image division grid width, the original image width, and the image width of the input lane line detection model, represents the initial set of vertical coordinates for dividing the grid, k ∈(0, j ).

8. A lane line detection method according to claim 1 or 7, characterized in that: In step S4, the lane line coordinate points in the middle part of the lane line are selected for fitting processing, specifically: 1 / 5 of the lane line coordinate points in the two ends of the lane line are removed, and the retained lane line coordinate points are fitted by the least square method.

9. The lane line detection method according to claim 1, characterized in that: After obtaining the optimal fitting lane line in step S4, the process also includes realizing vehicle lane change recognition in combination with the optimal fitting lane line, specifically: presetting a vehicle lane change slope threshold, and detecting the slope of the output optimal fitting lane line in real time. If the slope of the optimal fitting lane line does not exceed the preset vehicle lane change slope threshold, it is recognized that the vehicle has not changed lanes. If the slope of the best-fit lane line exceeds the preset vehicle lane change slope threshold, it is recognized that the vehicle has changed lanes.

Images