Multimodal Lane Marking Detection for Long-Range 3D Mapping
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional lane detection systems face challenges in accurately detecting lane markings due to sensor limitations, calibration issues, and maintaining continuity over long distances, especially in varying environmental conditions and complex road scenarios.
Innovation Solution
A system and method that fuse multimodal sensor data from cameras and LIDAR devices to generate accurate lane marking maps, using a convolutional neural network for image analysis and LIDAR point cloud processing, along with vehicle metrics, to address sensor limitations and calibration, and introduce a sub-pixel linearly decreasing function for perspective projection, enabling detection of solid, dotted, and non-traditional lane separators.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If camera-based visual information is used for lane detection, then the system can detect lane markings, but the detection accuracy deteriorates in varying environmental conditions, weather conditions, and lighting conditions
Solution Approach 1:
The patent combines camera-based visual information with LIDAR-based 3D point cloud data to create a multimodal lane detection system. The camera captures 2D images while the LIDAR captures 3D spatial information, and their fusion compensates for the weaknesses of each individual sensor modality, maintaining detection accuracy across varying environmental conditions
Solution Approach 2:
The patent transforms 2D image data from the camera into 3D space by integrating it with LIDAR point cloud data. This dimensional transformation allows the system to leverage both the visual features from the camera and the accurate depth information from LIDAR, improving lane marking detection robustness in adverse conditions
2Adaptability or versatility
If conventional single-sensor systems are used, then the device complexity is low, but the adaptability to different lane marking types and environmental conditions deteriorates
Solution Approach 1:
The patent creates a universal lane detection system that can handle multiple lane marking types (solid, dashed, different colors, non-traditional separators) and various environmental conditions by integrating multiple sensor modalities. The system processes both 2D image data and 3D point cloud data, making it adaptable to diverse scenarios while managing complexity through unified processing architecture
3Area of stationary object
If lane detection is performed over long distances, then the coverage area increases, but the continuity and smoothness of lane marking detection deteriorates
Solution Approach 1:
The patent merges 2D lane detection results from the camera with 3D lane detection results from the LIDAR system. This combination provides complementary information that maintains detection continuity over long distances, as the 3D point cloud data helps bridge gaps and maintain smooth lane trajectories across extended coverage areas
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
The solution achieves centimeter-level accuracy in lane marking detection over hundreds of miles, even in adverse conditions, by integrating multimodal sensor data to improve robustness and adaptability, ensuring continuous and smooth lane marking detection across diverse environments.
Implementation Method 1
receiving point cloud data from a distance and intensity measuring device mounted on the vehicle
Implementation Method 2
LIDAR point cloud data
Implementation Method 3
using a convolutional neural network for image analysis
Data Source
AI summary
A system and method for large-scale lane marking detection using multimodal sensor data are disclosed. A particular embodiment includes: receiving image data from an image generating device mounted on a vehicle; receiving point cloud data from a distance and intensity measuring device mounted on the vehicle; fusing the image data and the point cloud data to produce a set of lane marking points in three-dimensional (3D) space that correlate to the image data and the point cloud data; and generating a lane marking map from the set of lane marking points.