An improved lane line intelligent detection method based on UFSA algorithm

Abstract

The present invention discloses an improved lane line intelligent detection method based on the UFSA algorithm, including obtaining a lane line image, preprocessing the lane line image, and obtaining a lane line data set; dividing the lane line data set into independent and non-repetitive training sets, verification sets, and test sets; labeling and classifying each lane line image in the training set and the verification set; extracting features from each lane line image in the training set, the verification set, and the test set; improving the Resnet18 network model adopted by the UFSA algorithm, training and verifying the improved Resnet18 network model using the training set and the verification set, and obtaining a trained lane line detection model; using the trained lane line detection model to detect the test set and obtain the detection result. The method of the present invention realizes accurate positioning and lane line detection, effectively alleviates the urban traffic burden and traffic accidents, and assists in automatic driving to ensure driving safety.

Description

Technical Field

[0001] The invention relates to an intelligent lane line detection method based on an improved UFSA algorithm, and belongs to the technical field of lane line detection. Background Art

[0002] With the acceleration of urbanization and the increase in the number of cars, the burden of urban traffic is increasing, traffic accidents are frequent, and people's driving pressure is doubling. In order to effectively alleviate the pressure of traffic, avoid the increase of vehicle road accidents, and ensure driving safety, the research and development of vehicle driving assistance systems to help drivers has been booming worldwide in recent years. These systems obtain input from sensors on the vehicle and use some form of feedback signals output by the system to guide the driver to drive safely. Lane departure warning and lane keeping, lane change and forward collision warning, adaptive cruise control and blind spot detection systems all belong to the category of driving assistance systems. Lane marking detection is the core part of these systems. Therefore, the study of lane line detection has great theoretical value and practical significance.

[0003] More accurately and effectively detecting lane lines from images has been a widely concerned issue for researchers in recent years, and a lot of research has been done on lane line detection methods. Currently, there are two mainstream methods for lane detection, namely traditional image detection and processing methods and deep segmentation methods.

[0004] The traditional image detection processing method - deep segmentation UFSA algorithm has many problems, such as: it is difficult to obtain too rich image information, it is difficult to capture long-distance context information, and it may lose boundary information. Since the lane detection application scenario is extremely sensitive to lane boundary information, it not only needs to pay attention to the geometric shape of the object, but also needs to pay attention to edge information such as color, texture and lighting. Therefore, the traditional image processing method is difficult to apply to the special scenario of lane detection. Summary of the invention

[0005] The technical problem to be solved by the present invention is to provide an intelligent lane line detection method based on the improved UFSA algorithm, improve the UFSA algorithm so that the improved algorithm positioning and classification are more accurate, alleviate the traffic burden and traffic accidents to a certain extent, and ensure driving safety.

[0006] The present invention adopts the following technical solutions to solve the above technical problems:

[0007] An improved lane line intelligent detection method based on the UFSA algorithm, the lane line intelligent detection method comprising the following steps:

[0008] Step 1, obtain a lane line image from the benchmark dataset CULane, and preprocess the lane line image to obtain a lane line dataset;

[0009] Step 2: Divide the lane line dataset into independent and non-repetitive training sets, validation sets, and test sets;

[0010] Step 3: label and classify the lane line images in the training set and the validation set;

[0011] Step 4, extracting features from each lane line image in the training set, validation set, and test set to obtain features of each lane line image in the training set, validation set, and test set;

[0012] Step 5: Improve the Resnet18 network model used by the UFSA algorithm, that is, replace the Softmax loss function with the L-Dice loss function, and use the training set and validation set to train and validate the improved Resnet18 network model to obtain a trained lane line detection model;

[0013] Step 6: Use the trained lane detection model to detect the test set and obtain the detection results.

[0014] As a preferred solution of the present invention, the benchmark dataset CULane in step 1 contains a total of 133,235 images, including 88,880 training images, 9,675 verification images, and 34,680 test images.

[0015] As a preferred solution of the present invention, the lane line image is preprocessed in step 1 to obtain a lane line dataset, specifically: the lane line image is formatted into a set size to obtain a lane line dataset.

[0016] As a preferred solution of the present invention, in step 2, a random sampling method is used to divide the lane line data set.

[0017] As a preferred solution of the present invention, in step 3, the lane line image is labeled and classified using a target detection and labeling tool.

[0018] As a preferred solution of the present invention, the loss function of the improved Resnet18 network model described in step 5 is defined as:

[0019] L total =L cls +αL str +β(μL seg +γL L-Dice )

[0020] Among them, L total Represents the overall loss function, Lcls represents the classification loss, L str represents the structural loss, L seg and L L-Dice All represent segmentation losses, α, β, μ and γ all represent loss coefficients, μ is set to 0.7, and γ is set to 0.3.

[0021] Compared with the prior art, the present invention adopts the above technical solution and has the following technical effects:

[0022] 1. The present invention uses the benchmark data set to train the improved Resnet18 network model to obtain an intelligent lane line detection model, which can quickly locate the lane line. It is more efficient than previous methods and saves a lot of time and labor costs.

[0023] 2. The L-Dice loss function is added to the improved Resnet18 network model of the present invention. The model pays more attention to edge information, such as color, texture, and lighting, making detection and classification more accurate.

[0024] 3. The L-Dice loss function is added to the improved Resnet18 network model of the present invention, so that the feature maps of different layers are output more accurately, the output data is optimized, and the positioning accuracy is improved.

[0025] 4. The method of the present invention can effectively alleviate the increasingly heavy urban traffic burden and frequent traffic accidents, effectively reduce the pressure of people driving vehicles, and at the same time, ensure driving safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 It is an architecture diagram of an improved lane line intelligent detection method based on the UFSA algorithm of the present invention;

[0027] Figure 2 It is a schematic diagram of Dice Loss proposed in the present invention;

[0028] Figure 3 This is a simplified diagram of the UFSA network structure;

[0029] Figure 4 It is a simplified diagram of the improved UFSA network structure;

[0030] Figure 5 2 are effect diagrams of lane line detection before and after improvement of an embodiment of the present invention, wherein (a) is before improvement and (b) is after improvement. DETAILED DESCRIPTION

[0031] The embodiments of the present invention are described in detail below, and examples of the embodiments are shown in the accompanying drawings. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be interpreted as limiting the present invention.

[0032] The present invention discloses an improved lane line intelligent detection method based on the UFSA (Ultra Fast Structure-aware) algorithm. In view of the disadvantage that the original algorithm loses boundary information during lane line detection, the method mainly includes improving and fusing the Softmax loss function of the original auxiliary segmentation branch into the L-Dice (Lane Dice Loss) loss function, which makes the algorithm pay more attention to the lane boundary information. The effectiveness of the above improvement is verified by ablation experiments, and the improvement does not require any additional calculation.

[0033] like Figure 1 As shown, the present invention proposes an improved lane line intelligent detection method based on the UFSA algorithm, which specifically includes the following steps:

[0034] Step 1, obtain a lane line image from the benchmark dataset CULane, and preprocess the lane line image to obtain a lane line dataset;

[0035] Lane line image extraction comes from the CULane benchmark dataset. The dataset annotation is stored in a json file using the center dot format. The annotation of a road image is a tuple. The CULane dataset is a large-scale challenging dataset used for lane detection academic research. It contains a total of 133,235 images, divided into 88,880 training images, 9,675 verification images, and 34,680 test images. Preprocessing refers to formatting the lane line image to a set size to obtain a lane line dataset.

[0036] Step 2: Divide the lane line dataset into independent and non-repetitive training sets, validation sets, and test sets; random sampling can be used for division.

[0037] Step 3: Label and classify each lane line image in the training set and the validation set; here, the target detection and labeling tool can be used to label and classify the lane line images.

[0038] Step 4: Perform feature extraction on each lane line image in the training set, validation set, and test set to obtain the features of each lane line image in the training set, validation set, and test set.

[0039] Step 5: Improve the Resnet18 network model used by the UFSA algorithm, that is, improve the Softmax loss function of the original auxiliary segmentation branch to the L-Dice loss function, and use the training set and validation set to train and validate the improved Resnet18 network model to obtain a trained lane line detection model;

[0040] Step 6: Use the trained lane detection model to detect the test set and obtain the detection results.

[0041] A straightforward solution to boundary detection is to treat it as a semantic segmentation problem. In the annotation, the boundary is simply marked as 1 and other areas are marked as 0, that is, it is expressed as a binary semantic segmentation problem with binary cross entropy as the loss function. However, cross entropy has two limitations: the label distribution is highly unbalanced and the cross entropy loss of adjacent pixels at the boundary is difficult to determine. The cross entropy loss value only considers the loss in a microscopic sense, rather than the global sense.

[0042] Dice Loss can solve the above problems very well. This loss originated in the 1940s and is used to measure the similarity between two samples. The schematic diagram is as follows Figure 2 It was applied to computer vision by Miller et al. and used for 3D medical image segmentation in 2016. Its mathematical expression is:

[0043]

[0044] The above formula shows the dice coefficient equation, where p i and g i They represent the predicted pixel value and the true value, which are 0 or 1. In the lane detection scenario, it indicates whether the pixel is a boundary. If it is, the value is 1, and if it is not, the value is 0. The denominator is the total boundary pixel sum of the predicted value and the true value, and the numerator is the total boundary pixel sum of the correct prediction. i and g i The loss function increases only when the values ​​(both values ​​are 1) match. In this invention, Dice Loss is defined as L-Dice (Lane Dice Loss), and the segmentation loss at this time is defined as L L-Dice .

[0045] The original loss function is defined as:

[0046] L total =L cls +αL str +βL seg

[0047] L seg Corresponding to the segmentation loss, L str Corresponding structural loss, L cls Corresponding to classification loss, α and β are loss coefficients.

[0048] Then, the overall loss function is redefined as:

[0049] L total =L cls +αLstr +β(μL seg +γL L-Dice )

[0050] Among them, the segmentation loss L seg The coefficient μ is set to 0.7, and the segmentation loss L L-Dice The coefficient γ is set to 0.3.

[0051] Figure 3 and Figure 4 They are the traditional UFSA network structure and the improved UFSA network structure respectively.

[0052] For lane line detection, the CULane dataset released by the Chinese University of Hong Kong was used for relevant experiments and research. Most of the lane lines are obvious and clear, and the lane lines have little curvature.

[0053] As shown in Table 1, the CULane benchmark dataset is introduced in detail.

[0054] Table 1 Dataset description

[0055]

[0056] The dataset is divided into three parts: training set, validation set and test set. In the CULane dataset, the number of training iterations is set to 50; the batch size is 32, and the initial basic learning rate is 4e -4 , where momentum and weight decay are set to 0.9 and 0.00055 respectively. The training error and validation error use mean square error. At the end of training, the training error converges to 0.0044 and the validation error converges to 0.0045.

[0057] Precision and recall are the two most commonly used evaluation indicators for evaluating target detection and recognition algorithms. Precision is used to measure the proportion of the output prediction results that are correctly detected as true values, while recall is used to measure the proportion of the output prediction results that should be correctly detected.

[0058] The calculation formulas for precision and recall are:

[0059]

[0060]

[0061] TP is a true positive example, which means that the model correctly predicts the positive class samples as positive; TN is a true negative example, which means that the model correctly predicts the negative class samples as negative; FP is a false positive example, which means that the negative class samples are incorrectly predicted as positive; FN is a true negative example, which means that the negative class samples are incorrectly predicted as negative.

[0062] The lane line detection algorithm used is a fusion algorithm based on deep learning segmentation algorithm and traditional method optimization, so the image segmentation evaluation index is used to evaluate the detection results at the pixel level. F-measure, also known as F1 score, is the weighted harmonic average of precision and recall. It is used to judge the quality of the classification model. It is determined by two factors: precision and regression rate. The calculation formula is:

[0063]

[0064] When the β parameter is set to 1, the F-value function is set to the common F1, which combines the results of precision and recall. The higher the F1, the better the segmentation model.

[0065] The present invention is modified based on the Resnet-18 network model, adds L-Dice loss, and conducts experiments, as shown in Table 2.

[0066] Table 2 Comparison of the impact of UFSA algorithm improvements on model performance

[0067] experiment L-Dice F1 score / % Experiment 1 68.4 Experiment 2 √ 69.3

[0068] Table 2 shows the contents of the experimental improvements of the present invention, where Experiment 1 is the method of the original UFSA algorithm, and the corresponding F1 score is 68.4%; Experiment 2 is the detection result based on the original UFSA algorithm with L-Dice loss added, and the corresponding F1 score is 69.3%, which is 0.9% higher than the accuracy of the original UFSA. This experiment proves the effectiveness of the classification loss L-Dice, and alleviates the problems of losing important information when extracting lane features through network convolution and pooling, and failing to obtain richer and long-distance contextual information when the receptive field is large.

[0069] The experiments of this improved part are based on the public dataset CULane dataset, and a large number of experimental verifications have been carried out. Table 3 shows the recognition accuracy of 9 scenes based on different algorithms of the CULane dataset. The detection accuracy of the present invention based on Resnet-18 is shown in the table. The 9 scenes are: Normal, Crowded, Night, No-line, Shadow, Arrow, Dazzlelight, Curve, Crossroad. The corresponding accuracies of the original algorithm in Resnet-18 are: 87.7%, 66.0%, 62.1%, 40.2%, 62.8%, 81.0%, 58.4%, 57.9%, 1743; the corresponding accuracies of the present invention experiments are: 89.1%, 68.0%, 64.1%, 42.1%, 63.4%, 83.1%, 58.7%, 59.4%, 2336. After the improvement, the accuracy of the first 8 scenes was improved by 1.4%, 2.0%, 2.0%, 1.9%, 0.6%, 2.1%, 0.3%, and 1.5% respectively. The detection and recognition accuracy of the 9th scene was lower, and the number of misdetected lane lines was more than before the improvement, but the overall detection effect was good. The original Resnet-18 comprehensive detection accuracy was 68.4%, while the improved one was 69.9%, an increase of 1.5%; at the same time, the detection speed was also significantly faster than before, from 322.5FPS to 301.7FPS.

[0070] Table 3. Recognition accuracy of 9 scenes in CULane dataset. The test threshold is set to 0.5, the image resolution is set to 1640×590, and “-” indicates that the result is not available.

[0071]

[0072] Figure 5 The figures are effect diagrams of lane line detection before and after the improvement of the embodiment of the present invention, wherein (a) is before the improvement and (b) is after the improvement. The intelligent lane line detection method improved based on the UFSA algorithm of the present invention aims to standardize driving, improve the standardization of illegal driving, reduce traffic accidents caused by irregular driving, and improve the effect of lane line detection. The invention can be applied to different detection scenarios and can give the specific location of the lane line. The invention improves driving safety to a certain extent, reduces the growth rate of traffic accidents, and realizes intelligent detection.

[0073] The above embodiments are only for illustrating the technical idea of ​​the present invention, and cannot be used to limit the protection scope of the present invention. Any changes made on the basis of the technical solution in accordance with the technical idea proposed by the present invention shall fall within the protection scope of the present invention.

Claims

1. An improved lane line intelligent detection method based on the UFSA algorithm, characterized in that: The lane line intelligent detection method comprises the following steps: Step 1, obtain a lane line image from the benchmark dataset CULane, and preprocess the lane line image to obtain a lane line dataset; Step 2: Divide the lane line dataset into independent and non-repetitive training sets, validation sets, and test sets; Step 3: label and classify the lane line images in the training set and the validation set; Step 4, extracting features of each lane line image in the training set, validation set, and test set, respectively, to obtain features of each lane line image in the training set, validation set, and test set; Step 5: Improve the Resnet18 network model used by the UFSA algorithm by defining Dice Loss as L-Dice and defining the corresponding segmentation loss as L L-Dice , replace the Softmax loss function with the L-Dice loss function, use the training set and validation set to train and validate the improved Resnet18 network model, and obtain the trained lane line detection model; Step 6: Use the trained lane detection model to detect the test set and obtain the detection results.

2. The lane line intelligent detection method based on the improved UFSA algorithm according to claim 1 is characterized in that: The benchmark dataset CULane described in step 1 contains a total of 133,235 images, including 88,880 training images, 9,675 verification images, and 34,680 test images.

3. The lane line intelligent detection method based on the improved UFSA algorithm according to claim 1 is characterized in that: The lane line image is preprocessed in step 1 to obtain a lane line data set, specifically, the lane line image is formatted into a set size to obtain a lane line data set.

4. The lane line intelligent detection method based on the improved UFSA algorithm according to claim 1 is characterized in that: In step 2, the lane line data set is divided by random sampling.

5. The lane line intelligent detection method based on the improved UFSA algorithm according to claim 1 is characterized in that: In step 3, the lane line image is labeled and classified using a target detection and labeling tool.

6. The lane line intelligent detection method based on the improved UFSA algorithm according to claim 1 is characterized in that: The loss function of the improved Resnet18 network model described in step 5 is defined as: L total =L cls +αL str +β(μL seg +γL L-Dice ) Among them, L total Represents the overall loss function, L cls represents the classification loss, L str represents the structural loss, L seg and L L-Dice All represent segmentation losses, α, β, μ and γ all represent loss coefficients, μ is set to 0.7, and γ is set to 0.3.

Images