LiDAR-Camera Obstacle Labeling for Vegetation Collidability
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Autonomous vehicles face challenges in distinguishing between different types of obstacles, particularly vegetation, using LiDAR sensors, which can lead to misclassification and inefficient navigation, including over-braking and potential rear-end collisions.
Innovation Solution
A method and system that combines LiDAR and camera data to perform factor queries, including color, shape, and movement analysis, to accurately label obstacles as collidable or non-collidable, enabling the vehicle to adjust its actions such as speed and trajectory accordingly.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If LiDAR is used to detect obstacles, then the presence and absence of objects can be detected, but vegetation is misclassified as vehicles due to similar point-cloud shapes
Solution Approach 1:
The patent combines LiDAR point cloud data with camera image data to create a multi-modal sensing system. The LiDAR provides precise depth and spatial information while the camera provides visual texture and color information. By merging these data sources, the system can distinguish vegetation from vehicles more accurately than LiDAR alone, resolving the classification ambiguity caused by similar point-cloud shapes.
Solution Approach 2:
The patent introduces an intermediary classification system that processes both LiDAR and camera data together. This intermediary system uses the complementary information from both sensors to disambiguate obstacle types. The camera acts as an intermediary that provides visual cues (color, texture) to differentiate vegetation from vehicles when LiDAR point clouds are ambiguous.
2Reliability
If LiDAR classifies vegetation as vehicles, then the AV may stop for vegetation, but this causes over-braking which degrades ride quality and introduces complexities in motion planning
Solution Approach 1:
The patent performs preliminary classification of obstacles as collidable or non-collidable before executing motion planning. By pre-labeling vegetation as non-collidable based on combined LiDAR and camera analysis, the system prepares classification results in advance, allowing the motion planning module to operate more efficiently without needing to handle ambiguous vegetation cases during real-time planning.
3Reliability
If the AV stops for vegetation to prevent collision, then safety is improved, but the AV runs slower and more inefficiently and may increase chances of being rear-ended
Solution Approach 1:
The patent applies different collision avoidance strategies to different obstacle types based on their local properties. Vegetation is classified as non-collidable and allows controlled collision, while vehicles and pedestrians are classified as collidable and trigger avoidance maneuvers. This local differentiation enables the AV to brush aside vegetation without stopping, maintaining navigation efficiency while still preventing collisions with dangerous obstacles.
4Productivity
If the AV collides with vegetation to maintain progress, then navigation efficiency is improved, but objects such as pedestrians may be hidden by vegetation and collision becomes dangerous
Solution Approach 1:
The patent performs preliminary scanning and classification of vegetation areas before the AV proceeds. By using both LiDAR and camera data in advance, the system identifies and flags potential hidden dangers behind vegetation. This preliminary action allows the AV to maintain navigation efficiency by colliding with safe vegetation while avoiding areas where vegetation might conceal pedestrians or other dangerous objects.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
Improves the classification of obstacles, reducing misclassification and enhancing navigation efficiency by accurately determining whether to interact with vegetation, thereby preventing unnecessary braking and improving safety and efficiency.
Implementation Method 1
A LIDAR sensor is configured to emit light, which strikes material (e.g., objects) within the vicinity of the LiDAR sensor. Once the light contacts the material, the light is deflected. Some of the deflected light bounces back to the LiDAR sensor.
Data Source
AI summary
Systems and methods for detecting and labeling a collidability of obstacles within a vehicle environment are provided. The method may comprise generating one or more data points from one or more sensors coupled to a vehicle. The method may comprise, using a processor, detecting one or more obstacles within a LiDAR point cloud, generating a patch for each of the one or more detected obstacles, projecting the LiDAR point cloud into the image, performing a factor query on an image for each of the one or more detected obstacles, for each of the one or more detected obstacles, based on the factor query, determining a label for the obstacle, and, for each of the one or more detected obstacles, labeling the obstacle with the label. The label may indicate whether each of the one or more detected obstacles is collidable and not non-collidable.