LiDAR Camera Fusion for Autonomous Obstacle Classification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Autonomous vehicles face challenges in distinguishing between different types of obstacles, particularly vegetation and pedestrians, due to similarities in LiDAR point-cloud shapes, leading to misclassification and inefficient navigation.
Innovation Solution
A method and system that combines LiDAR and camera data to perform color and shape queries on detected obstacles, accurately labeling them as vegetation or pedestrians, and determining appropriate vehicle actions such as speed adjustments or trajectory changes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If LiDAR is used to detect obstacles, then the presence and absence of objects can be detected, but vegetation is misclassified as vehicles due to similar point-cloud shapes
Solution Approach 1:
The patent combines LiDAR point-cloud data with camera image data to create a multi-modal detection system. The LiDAR provides precise depth and spatial information while the camera provides color and texture information, allowing the system to distinguish vegetation from vehicles by fusing these complementary data sources rather than relying on LiDAR shape analysis alone
Solution Approach 2:
The patent introduces an intermediary classification process that takes LiDAR-detected obstacles and uses camera images as an additional verification layer. The camera acts as an intermediary sensor that provides color information to disambiguate cases where LiDAR alone cannot distinguish between vegetation and vehicles based on shape alone
2Reliability
If LiDAR classifies vegetation as vehicles, then safety margins are increased, but unnecessary braking occurs degrading ride quality
Solution Approach 1:
The system implements feedback by continuously monitoring obstacle classifications and using camera color information to verify LiDAR detections. When vegetation is incorrectly classified as a vehicle, the camera's color data provides feedback to correct the classification, preventing unnecessary braking actions and maintaining ride quality while preserving safety margins through accurate real-time classification
3Reliability
If LiDAR detects all objects conservatively, then collision avoidance is improved, but navigation efficiency decreases and rear-end collisions increase
Solution Approach 1:
The patent applies local quality by using different detection strategies for different types of obstacles. The system uses conservative detection for potential hazards while using camera color verification for vegetation identification, allowing efficient navigation through accurate local classification of each detected object rather than applying uniform conservative detection to all objects
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
Improves the accuracy of obstacle classification, reducing the likelihood of unnecessary braking and collisions, and enabling more efficient navigation by distinguishing between collidable and non-collidable obstacles.
Implementation Method 1
A LiDAR sensor is configured to emit light, which strikes material (e.g., objects) within the vicinity of the LiDAR sensor. Once the light contacts the material, the light is deflected. Some of the deflected light bounces back to the LiDAR sensor.
Implementation Method 2
The LiDAR sensor is configured to measure data pertaining to the light bounced back (e.g., the distance traveled by the light, the length of time it took for the light to travel from and to the LiDAR sensors)
Data Source
AI summary
Systems and methods for detecting and labeling one or more obstacles within a vehicle environment are provided. The method may comprise generating one or more data points from one or more sensors coupled to a vehicle and, using a processor, detecting one or more obstacles within a LiDAR point cloud, generating a patch for each of the one or more detected obstacles, projecting the LiDAR point cloud into an image, performing a color query on the image for each obstacle, performing a shape query on the image for each of the one or more detected obstacles, for each of the one or more detected obstacles, determining a label for the obstacle based on one or more of the color query and the shape query and labeling the obstacle with the label. The label may indicate whether each of the one or more detected obstacles is a piece of vegetation and not a pedestrian.