Multitask DNN for Lane Detection via Segmentation and Keypoint Regression
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing lane and roadway detection methods face challenges in accurately recognizing boundaries, especially when markings are missing or obscured by dynamic objects, leading to difficulties in extracting reliable lane and road information.
Innovation Solution
A multitask deep neural network (DNN) architecture comprising a convolutional neural network (CNN) encoder and two decoders for semantic segmentation and key point regression is used to provide pixel-precise classification and sub-pixel-precise boundary detection, with redundancy for estimating detection confidence.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If semantic segmentation is used for lane and roadway detection, then classification performance is improved, but boundary extraction becomes ambiguous and unreliable
Solution Approach 1:
The patent segments the detection task into two independent parts: semantic segmentation for classifying drivable areas and keypoint regression for detecting boundaries. This segmentation allows each task to be optimized independently, resolving the contradiction between classification performance and boundary extraction reliability.
Solution Approach 2:
The patent introduces keypoint regression as an intermediary component that bridges the gap between semantic segmentation results and actual boundary detection. The keypoint regression module takes the semantic segmentation output and refines it by explicitly detecting boundary keypoints, thereby recovering the boundary information that was lost in pure semantic segmentation.
2Adaptability or versatility
If regression approach is used for boundary detection, then boundary interpolation capability is improved, but detection reliability becomes difficult to assess
Solution Approach 1:
The patent implements a feedback mechanism where the keypoint regression output is fed back to the semantic segmentation module. This feedback loop allows the system to continuously refine and adjust its detection results, improving both interpolation capability and reliability assessment by comparing and validating the two detection approaches against each other.
3Device complexity
If single-task neural network is used, then system complexity is reduced, but detection reliability in challenging scenarios deteriorates
Solution Approach 1:
The patent creates a multi-functional neural network system where a single semantic segmentation module serves multiple purposes: classifying drivable areas, providing boundary information, and supporting keypoint regression. This multi-functionality approach maintains system efficiency while improving detection reliability through redundant information sources and cross-validation between tasks.
Data Source
Figure 1~2
Figure 3~4
Figure 5
AI summary
The invention relates to a method for lane detection (S1-S3) and roadway detection (1), wherein a multitask DNN architecture is used for detection and consists of at least one encoder and at least one first and one second decoder, the method comprising the following steps: - providing (VS1) an input image by means of an optical detection device, - filtering (VS2) the input image by means of the encoder, - generating (VS3) a first representation of the lane (S1-S3) and/or roadway (1) by the encoder, - forwarding (VS4) the first representation to the first and the second decoder, - processing (VS5) the first representation in the first and the second decoder, - outputting (VS6) two further different representations of the first and second decoder, - combining (VS7) the two representations of the first and the second decoder, - outputting (VS8) identified lanes (S1-S3) and lane markings and also the roadway (1).