Dense lane line detection method based on correlation discrimination

Abstract

The invention discloses a dense lane line detection method based on correlation discrimination. The method comprises the steps that a plurality of lane line diagrams are shot and collected through a camera, and all the lane line diagrams are marked to obtain a lane line data set; processing the lane line data set by adopting a lane line category label processing method to obtain a lane line category label data set; constructing a dense lane line detection model, and inputting the obtained lane line category label data set into the dense lane line detection model for training; and finally, inputting a to-be-detected lane line graph into the trained dense lane line detection model for detection to obtain the positions of all lane lines in the lane line graph. According to the invention, the lane line category label processing method is provided to process the lane line data set, and the provided correlation discrimination module can effectively learn the data processed by the lane line category label processing method, so that the relationship between the lane lines can be effectively identified, and the detection effect of the dense lane lines is improved.

Description

Technical Field

[0001] The present invention relates to the field of computer vision, and particularly to a dense lane line detection method based on correlation discrimination. Background Art

[0002] Lane line detection is an important task in the field of computer vision. It is a field that promotes and develops mutually with deep learning and can be applied to autonomous driving or assisted driving to provide information about road lane lines, thereby helping intelligent vehicles better locate their positions.

[0003] Lane line detection is a very challenging task in computer vision. However, the existing work has not explored dense lane line detection enough. Dense lane lines refer to road configurations characterized by multiple paths converging or diverging in a y-shaped or fork-shaped manner. These types of lanes usually appear in complex driving scenarios, such as intersections, highway exits, and urban environments. Generally, a non-maximum suppression module is introduced in the post-processing of lane line detection to suppress redundant detection results. However, since the adjacent lane lines of dense lane lines are also relatively close in distance, if NMS is introduced, it is easy to suppress the results that should be detected. On the contrary, removing the lane lines easily introduces some redundant results. Therefore, this type of dense lane line detection is relatively difficult. The paper "CondLaneNet: a Top-to-down Lane Detection Framework Based on Conditional Convolution" published at the international top conference ICCV2022 proposed a Recurrent Instance Module to detect dense lane lines. However, this method is not easy to parallel compute and is time-consuming. Summary of the Invention

[0004] To solve the deficiencies in the background art, the present invention provides a dense lane line detection method based on correlation discrimination. The present invention introduces a correlation discrimination module, which can well identify the relationship between lane lines, and then obtain high-precision detection results for dense lane lines.

[0005] The technical solution adopted by the present invention is as follows:

[0006] A dense lane line detection method based on correlation discrimination of the present invention includes the following steps:

[0007] S1. Collect a number of lane line images by camera shooting, annotate each lane line image to obtain a number of lane lines on each lane line image, and obtain a lane line dataset after annotating all lane line images.

[0008] For all lane line icons, it is only marked in the figure which ones are lane lines, without numbering the lane lines or sorting the categories of the lane lines. The obtained lane line dataset is in the same form as the existing publicly available lane line dataset.

[0009] S2. Process all lane lines in each lane line diagram of the obtained lane line dataset using the lane line category label processing method to obtain a lane line category label dataset.

[0010] S3. Construct a dense lane line detection model, input the obtained lane line category label dataset into the dense lane line detection model for training, and obtain a trained dense lane line detection model.

[0011] S4. Input the lane line diagram to be measured into the trained dense lane line detection model for detection, and obtain the positions of all lane lines in the lane line diagram.

[0012] The lane line category label processing method is a category label method after correlation discrimination among all lane lines.

[0013] The lane line category label processing method specifically includes the following steps:

[0014] D1. For all lane lines obtained from each lane line diagram in the lane line dataset, perform distance processing on every two adjacent lane lines to obtain the distance between every two adjacent lane lines. The distance processing can use integral methods, etc.

[0015] D2. Perform traversal comparison processing on the obtained distances between every two adjacent lane lines. After the traversal comparison processing, several categories of lane lines and the number of lane lines in each category are obtained.

[0016] D3. Perform dense lane division processing based on the obtained several categories of lanes and the number of lane lines in each category to obtain each lane line diagram with labels.

[0017] The traversal comparison processing is specifically as follows: Traverse and compare the obtained distances between every two adjacent lane lines with a preset threshold distance: If the distance between two adjacent lane lines is less than or equal to the preset distance threshold, the two lane lines are of the same category of lane lines and the number of lane lines is updated. If the distance between two adjacent lane lines is greater than the preset distance threshold, the two lane lines are of two categories of lane lines and the number of lane lines in the two categories are updated respectively.

[0018] The dense lane division processing specifically includes the following steps:

[0019] F1. All lane lines in a category where the number of lane lines is equal to one are regarded as independent lane line categories, and all lane lines in all independent lane line categories are labeled with independent lane line labels.

[0020] F2. For each type of lane line with more than one lane, it is regarded as a type of dense lane line category respectively, and all the obtained dense lane line categories are sequentially labeled with the main dense lane line label; all the lane lines in each type of dense lane line category are sequentially labeled with the sub-dense lane line label.

[0021] The dense lane line detection model includes a detection module, a correlation discrimination module, and a suppression module; each lane line map is input into the detection module for detection processing to obtain a number of lane lines, all the lane lines are input into the correlation discrimination module for discrimination processing to obtain the category classification result of each lane line, and the category classification results of all the lane lines are then input into the suppression module for suppression processing to obtain the positions of all the lane lines in each lane line map, and the positions of all the lane lines in each obtained lane line map are used as the output of the dense lane line detection model. The correlation discrimination module includes a convolutional layer and a fully connected layer. All the lane lines obtained by the detection model are input into the convolutional layer for feature extraction to obtain feature data, and the result after the residual connection between the feature data and all the lane lines is input into the fully connected layer for classification processing to obtain the category classification result, and the obtained category classification result is then input into the suppression module. The detection module adopts the CLRNet model. The suppression module adopts the non-maximum suppression method NMS method.

[0022] The innovation of the present invention lies in proposing a lane line category label processing method to process the lane line data set, and the proposed correlation discrimination module can effectively learn the data processed by the lane line category label processing method, realizing the effective recognition of the relationship between lane lines and improving the detection effect of dense lane lines.

[0023] The beneficial effects of the present invention are:

[0024] 1. The present invention proposes a lane line category label processing method to process the lane line data set, and the proposed correlation discrimination module can effectively learn the data processed by the lane line category label processing method, realizing the effective recognition of the relationship between lane lines and improving the detection effect of dense lane lines.

[0025] 2. The present invention can be well applied to various lane line detection networks, and there is a great improvement in accuracy on the mainstream lane line detection data sets SDLane and Curvelanes, demonstrating the superiority of the algorithm. Description of the Drawings

[0026] Figure 1 It is the detection result diagram of the method of the present invention after removing the correlation discrimination module.

[0027] Figure 2This is the detection result graph of the method after removing the correlation discrimination module and suppression module of the method of the present invention.

[0028] Figure 3 This is the network structure diagram of the correlation discrimination module of the present invention. Detailed implementation manners

[0029] The following further describes the present invention in detail with reference to the drawings and embodiments. However, the present invention is not limited thereto. For those of ordinary skill in the art in this technical field, without departing from the principle of the present invention, several improvements and refinements can be made, and these improvements and refinements are also regarded as within the protection scope of the present invention. The content not described in detail in this specification belongs to the prior art well-known to those skilled in the art.

[0030] Embodiment 1 of the present invention is as follows:

[0031] S1. Collect several lane line images by camera shooting. After annotating each lane line image, several lane lines are obtained on each lane line image. After annotating all lane line images, a lane line data set is obtained.

[0032] All lane line images are annotated only to mark which are lane lines in the image, without numbering the lane lines or sorting the categories of the lane lines. The obtained lane line data set is in the same form as the publicly available lane line data set.

[0033] S2. Process all lane lines in each lane line image of the obtained lane line data set by using the lane line category label processing method to obtain a lane line category label data set.

[0034] The lane line category label processing method is a category label method after correlation discrimination among all lane lines.

[0035] The lane line category label processing method specifically includes the following steps:

[0036] D1. According to all lane lines obtained on each lane line image in the lane line data set, distance processing is performed on every two adjacent lane lines to obtain the distance between every two adjacent lane lines.

[0037] The distance processing can adopt integral methods, etc.

[0038] D2. The obtained distances between every two adjacent lane lines are subjected to traversal comparison processing. After the traversal comparison processing, several categories of lane lines and the number of lane lines in each category are obtained.

[0039] The traversal comparison process specifically involves traversing and comparing the distance between each adjacent pair of lane lines obtained with a preset threshold distance: If the distance between two adjacent lane lines is less than or equal to the preset distance threshold, the two lane lines are of the same type of lane line, the number of lane lines is updated, and the two lane lines are stored in the same type of lane line. If the distance between two adjacent lane lines is greater than the preset distance threshold, the two lane lines are of two types of lane lines, and the number of lane lines of the two types is updated separately.

[0040] In specific implementation, if a lane line is determined to be of the same type of lane line after the first comparison and processing with the lane line on the left, the number of lane lines of this type is updated to 2. When this lane line is later determined to be of the same type of lane line as the lane line on the right and the number of lane lines of this type is updated, this lane line is not counted repeatedly, and the number of lane lines of this type is updated to 3.

[0041] D3. Perform dense lane division processing based on the obtained several types of lanes and the number of lane lines in each type of lane line to obtain each lane line map with tags.

[0042] The dense lane division processing specifically includes the following steps:

[0043] F1. All lane lines with a lane line number equal to one are regarded as independent lane line categories, and all lane lines in all independent lane line categories are labeled with independent lane line tags.

[0044] F2. Each type of lane line with a lane line number greater than one is regarded as a type of dense lane line category, and all obtained dense lane line categories are sequentially labeled with dense lane line main tags; all lane lines in each type of dense lane line category are sequentially labeled with dense lane line sub-tags.

[0045] In specific implementation, all lane lines in all independent lane line categories are labeled with independent lane line tags, and they are all 0.

[0046] All obtained dense lane line categories are sequentially labeled with tags, which are 1, 2, 3, 4, 5... N.

[0047] All lane lines in each type of dense lane line category are sequentially labeled with dense lane line sub-tags. For example: all lane lines in the first type of dense lane line category are sequentially labeled with sub-tags, which are 1-1, 1-2, 1-3... 1-M.

[0048] S3. Build a dense lane line detection model in the computer, input the obtained lane line category label data set into the dense lane line detection model for training, and obtain a trained dense lane line detection model.

[0049] The dense lane line detection model includes a detection module, a correlation discrimination module, and a suppression module; each lane line map is input into the detection module for detection processing to obtain several lane lines, all lane lines are input into the correlation discrimination module for discrimination processing to obtain the category classification results of each lane line, and the category classification results of all lane lines are then input into the suppression module for suppression processing to obtain the positions of all lane lines in each lane line map. The positions of all lane lines in each lane line map obtained are used as the output of the dense lane line detection model.

[0050] In a specific implementation, if the category classification results obtained after the correlation discrimination module classifies a lane line map are: 0, 1, 1, 2, 0, and then the category classification results with 0, 1, 1, 2, 0 are input into the suppression module, the suppression module will suppress the same category to obtain 0, 1, 2, 0, and finally obtain a lane line map containing lane lines of 0, 1, 2, 0.

[0051] The correlation discrimination module includes a convolutional layer and a fully connected layer. All lane lines obtained by the detection model are input into the convolutional layer for feature extraction to obtain feature data, and the result of the residual connection between the feature data and all lane lines is input into the fully connected layer for classification processing to obtain the category classification results, and the obtained category classification results are then input into the suppression module. The present invention uses a correlation discrimination module to classify dense lane line categories, and its network structure is as Figure 3 shown. The correlation discrimination module first aggregates and enhances the features of adjacent lane lines for all sorted lane lines. Specifically, the features of each lane line will be feature-aggregated with the adjacent k lane lines, so that each feature instance can perceive the features of nearby lane lines. For the features after feature enhancement, the features of the original lane lines are added for residual connection. Finally, the classification results of each lane line are output through the fully connected layer. The detection module uses the CLRNet model. The suppression module uses the non-maximum suppression method NMS method.

[0052] The innovation of the present invention lies in proposing a lane line category label processing method to process the lane line data set, and the proposed correlation discrimination module can effectively learn the data processed by the lane line category label processing method, realizing the effective recognition of the relationship between lane lines and improving the detection effect of dense lane lines.

[0053] S4. Input the lane line map to be measured into the trained dense lane line detection model for detection to obtain the positions of all lane lines in the lane line map.

[0054] Embodiment 2

[0055] This embodiment uses the same scheme as Embodiment 1 to conduct experiments on the SDLane and CurveLanes data sets.

[0056] Table 1

[0057]

[0058] The first row of the SDLane dataset and the first row of the CurveLanes in Table 1 are the methods after removing the CDM (correlation discrimination module) and NMS (suppression module) of the method of the present invention;

[0059] As Figure 1 shown, it is the detection result diagram of the method after removing the CDM (correlation discrimination module) of the method of the present invention. This method will cause the lane lines pointed by the red arrow to be missed.

[0060] As Figure 2 shown, it is the detection result diagram of the method after removing the CDM (correlation discrimination module) and NMS (suppression module) of the method of the present invention. This method will cause the lane lines pointed by the red arrow to be repeatedly recognized.

[0061] It can be seen from Table 1 that the application of NMS or Soft-NMS shows different performances on the two datasets. When using NMS on the SDLane dataset, the performance drops significantly, while on the CurveLanes dataset, the performance improves. This indicates that using NMS may remove more dense lane lines than near-duplicate predictions on the SDLane dataset, while there are fewer dense lane lines on CurveLanes. On the other hand, the performance of using Soft-NMS on CurveLanes is not as good as that of NMS, which indicates that applying Soft-NMS may also introduce near-duplicate predictions. The present invention has achieved performance improvement on both datasets by adding CDM on the basis of using NMS, which is reflected in all evaluation metrics, proving the superiority of the invention.

Claims

1. A dense lane line detection method based on correlation discrimination, characterized in that: The following steps are involved: S1. Collect several lane line maps by camera shooting, annotate each lane line map to obtain several lane lines on each lane line map, and obtain a lane line dataset after annotating all lane line maps; S2. Processing all lane lines in each lane line map of the obtained lane line data set using a lane line category label processing method to obtain a lane line category label data set; S3, building a dense lane line detection model, inputting the obtained lane line category label data set into the dense lane line detection model for training, and obtaining a trained dense lane line detection model; S4. Input the lane line map to be tested into the trained dense lane line detection model for detection to obtain the positions of all lane lines in the lane line map.

2. The dense lane line detection method based on correlation discrimination according to claim 1 is characterized in that: The lane line category label processing method specifically includes the following steps: D1. According to all lane lines obtained on each lane line map in the lane line data set, distance processing is performed on each adjacent lane line to obtain the distance between each adjacent lane lines; D2. Perform traversal and comparison processing on the distance between each two adjacent lane lines, and obtain several types of lane lines and the number of lanes of each type of lane line after traversal and comparison processing; D3. Dense lane division is performed based on the obtained lane types and the number of lanes in each lane type to obtain each lane line map after labeling.

3. The dense lane line detection method based on correlation discrimination according to claim 2 is characterized in that: The traversal comparison process is specifically as follows: The distance between each adjacent lane line is compared with the preset threshold distance: If the distance between two adjacent lane lines is less than or equal to the preset distance threshold, the two lane lines are of the same type and the number of lane lines is updated; If the distance between two adjacent lane lines is greater than a preset distance threshold, the two lane lines are two types of lane lines and the lane line numbers of the two types of lane lines are updated respectively.

4. The dense lane line detection method based on correlation discrimination according to claim 2 is characterized in that: The dense lane division process specifically includes the following steps: F1. All lane lines of a class with a lane number equal to one are regarded as independent lane line categories, and all lane lines of all independent lane line categories are marked with independent lane line labels; F2. Treat each type of lane line with a lane number greater than one as a type of dense lane line category, and mark all the obtained dense lane line categories with dense lane line main labels in sequence; mark all lane lines in each type of dense lane line category with dense lane line sub-labels in sequence.

5. The dense lane line detection method based on correlation discrimination according to claim 1, characterized in that: The dense lane line detection model includes a detection module, a correlation judgment module and a suppression module; each lane line map is input into the detection module for detection processing to obtain a number of lane lines, all lane lines are input into the correlation judgment module for judgment processing to obtain the category classification result of each lane line, and the category classification results of all lane lines are input into the suppression module for suppression processing to obtain the positions of all lane lines in each lane line map, and the positions of all lane lines in each lane line map are obtained as the output of the dense lane line detection model.

6. The dense lane line detection method based on correlation discrimination according to claim 1, characterized in that: The correlation discrimination module includes a convolution layer and a fully connected layer. All lane lines obtained by the detection model are input into the convolution layer for feature extraction to obtain feature data. The feature data is then residually connected with all lane lines and the result is input into the fully connected layer for classification processing to obtain a category classification result. The obtained category classification result is then input into the suppression module.

7. The dense lane line detection method based on correlation discrimination according to claim 1 is characterized in that: The detection module adopts the CLRNet model; The suppression module adopts a maximum value suppression method.

Images