3D Lane Detection Training with Spatio-Temporal Consistency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing lane detection systems for autonomous vehicles rely heavily on annotated 3D data, which is expensive and error-prone, and struggle with accuracy in complex road environments, necessitating a cost-effective and reliable method for 3D lane detection.
Innovation Solution
A method for training an artificial neural network using unlabelled image data from vehicle-mounted cameras, employing spatio-temporal consistency loss to align and update model parameters based on matching lane pairs across images, leveraging GPS data for coordinate transformation and using a 3D-lanenet architecture for 3D lane detection.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If annotated 3D data is used for training lane detection models, then detection accuracy is improved, but annotation costs and time increase significantly
Solution Approach 1:
The system uses self-supervised learning where the model trains itself on unannotated data by generating pseudo-labels and applying spatio-temporal consistency constraints. The model leverages its own predictions across multiple time steps to create training signals, eliminating the need for manual 3D annotation while maintaining detection accuracy.
Solution Approach 2:
The system performs preliminary 2D lane detection and projection to generate initial 3D lane boundary predictions before final detection. This preliminary action creates a foundation that reduces the complexity of direct 3D annotation requirements, allowing the model to learn from unannotated data through iterative refinement.
2Measurement precision
If annotated 3D data is used for training lane detection models, then detection accuracy is improved, but annotation costs increase
Solution Approach 1:
The system employs self-supervised learning mechanisms where the model generates its own training data through pseudo-labeling and consistency constraints. By using unannotated video sequences and leveraging spatio-temporal relationships, the model eliminates expensive manual 3D annotation while achieving high detection accuracy through self-directed learning.
Solution Approach 2:
The system creates synthetic training data by projecting 2D lane detections into 3D space and using temporal copies of lane boundaries from adjacent frames. These copied and transformed predictions serve as pseudo-labels for training, replacing the need for expensive real 3D annotated data while maintaining detection performance.
3Ease of manufacture
If 2D lane detection methods are used, then annotation requirements are reduced, but 3D position accuracy deteriorates
Solution Approach 1:
The system transforms 2D lane detection predictions into 3D space by applying camera intrinsics, extrinsics, and ground plane projections. This dimensional transformation allows the model to work with easier-to-obtain 2D annotations while producing accurate 3D lane boundary positions through mathematical projection and spatio-temporal consistency enforcement.
Solution Approach 2:
The system performs preliminary 2D lane detection and coordinate transformation to world space before final 3D lane boundary prediction. This preliminary action in 2D space reduces annotation requirements, and the subsequent 3D projection with temporal consistency constraints restores the accuracy needed for reliable 3D position estimation.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
The present invention relates to a method for training an artificial neural network configured for 3D lane detection based on unlabelled image data from a vehicle-mounted camera. The method comprises generating, by means of the artificial neural network, a first set of 3D lane boundaries in a first coordinate system based a first image captured by the vehicle-mounted camera. The method further comprises generating, by means of the artificial neural network, a second set of 3D lane boundaries in a second coordinate system based on a second image captured by the vehicle mounted camera. The second image is captured at a later moment in time compared to the first image and the first image and the second image contain at least partly overlapping road portions. Further, the method comprises transforming at least one of the second set of 3D lane boundaries and the first set of 3D lane boundaries based on positional data associated with the first image and the second image, such that the first set of 3D lane boundaries and the second set of 3D lane boundaries have a common coordinate system. The method further comprises evaluating the first set of 3D lane boundaries against the second set of 3D lane boundaries in the common coordinate system in order to find matching lane pairs of the first set of 3D lane boundaries and the second set of 3D lane boundaries. Furthermore, the method comprises updating one or more model parameters of the artificial neural network based on a spatio-temporal consistency loss between the found matching lane pairs of the first set of 3D lane boundaries and the second set of 3D lane boundaries.