3D Lane Detection via Feature Map Projection and Sensor Fusion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current lane detection technologies in road scenes face challenges due to poor fidelity in detection and representation, especially when fusing sensor data from different modalities, which can impede the accurate operation of autonomous vehicles.
Innovation Solution
A multi-layer convolutional neural network is employed to generate feature maps from images of a road scene, projecting them onto a defined plane using homographic transformation, and applying convolutions to determine lane information, which is then used to adjust vehicle operation, incorporating data from multiple sensor modalities to enhance detection fidelity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If sensor data from different modalities is fused after lane detection, then multiple sensor types can be utilized, but detection fidelity deteriorates
Solution Approach 1:
The patent performs sensor fusion at the feature map level before lane detection rather than after detection. Multiple sensor modalities (camera, LIDAR, radar) are processed to generate feature maps that are then fused together in the neural network, allowing the system to leverage multiple sensors while maintaining high detection fidelity through early integration of sensor data
2Ease of manufacture
If heuristics are applied to combine road elements into lanes, then road elements can be processed, but detection fidelity deteriorates
Solution Approach 1:
The patent replaces heuristic-based lane combination methods with a deep learning approach. A multi-layer convolutional neural network automatically learns how to combine road elements into lanes through training on labeled data, eliminating the need for manual heuristic rules and achieving superior detection fidelity while maintaining processing capability
3Ease of operation
If lanes are detected separately in image domain, then detection can be performed, but representation fidelity deteriorates
Solution Approach 1:
The patent transforms the lane detection problem from 2D image domain to 3D spatial representation. The neural network outputs not only lane positions in the image plane but also depth information, creating three-dimensional lane representations that maintain geometric accuracy and improve representation fidelity while keeping the detection operation straightforward
Data Source
AI summary
A vehicle, system for operating a vehicle and method of navigating a vehicle. The system includes a sensor and a multi-layer convolutional neural network. The sensor generates an image indicative of a road scene of the vehicle. The multi-layer convolutional neural network generates a plurality of feature maps from the image via a first processing pathway, projects at least one of the plurality of feature maps onto a defined plane relative to a defined coordinate system of the road scene to obtain at least one projected feature map, applies a convolution to the at least one projected feature map in a second processing pathway to obtain a final feature map, and determines lane information from the final feature map. A control system adjusts operation of the vehicle using the lane information.