Road Surface Hazard Sensing With LiDAR-Image Transformer Fusion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current sensor technologies struggle to accurately detect hazards and road surface profiles at far distances due to limitations in LiDAR, RADAR, and camera-based solutions, particularly in varying weather conditions, leading to incomplete and unreliable representations.
Innovation Solution
Employing a transformer to fuse image and LiDAR data for hazard and surface detection, using sampled features to decode representations of hazards and surface features, and generating corresponding ground truth data for autonomous vehicles.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If LiDAR is used for hazard detection, then measurement precision is improved at close range, but reliability deteriorates at far distances and in wet weather conditions
Solution Approach 1:
The patent combines multiple sensor types (LiDAR, RADAR, cameras) into a fused sensor system that leverages the strengths of each sensor to compensate for their individual weaknesses, thereby improving both measurement precision and reliability across varying distances and weather conditions
Solution Approach 2:
The patent creates a composite sensing system that integrates data from different sensor modalities, similar to how composite materials combine different substances to achieve properties that individual materials cannot provide alone, enabling robust hazard detection in diverse environmental conditions
2Reliability
If RADAR is used for long-range detection, then reliability is improved in various weather conditions, but measurement precision deteriorates due to lower resolution
Solution Approach 1:
The patent fuses RADAR data with high-resolution camera and LiDAR data, allowing the system to maintain the reliability and long-range detection capabilities of RADAR while compensating for its lower resolution through complementary high-precision sensors
Solution Approach 2:
The patent applies different sensing modalities to different spatial regions and detection tasks, using RADAR for long-range preliminary detection and high-resolution sensors for detailed classification of identified hazards, optimizing both reliability and precision
3Measurement precision
If camera-only solutions are used for hazard detection, then measurement precision is improved for visual data, but reliability deteriorates due to sensitivity to lighting conditions and poor depth perception
Solution Approach 1:
The patent integrates camera visual data with depth information from LiDAR and stereo vision systems, compensating for the camera's poor depth perception while maintaining its high-resolution visual capabilities through multi-sensor fusion
Solution Approach 2:
The patent uses active vision systems and depth estimation algorithms as intermediaries to bridge the gap between 2D camera images and 3D spatial understanding, enabling reliable depth perception by combining passive visual cues with active ranging measurements
4Reliability
If multiple sensor types are fused for comprehensive detection, then reliability is improved across various conditions, but device complexity increases
Solution Approach 1:
The patent designs a unified sensor fusion architecture that processes multiple sensor types through a common computational framework, allowing the system to handle diverse sensor inputs efficiently without requiring separate processing pipelines for each sensor type, thereby managing complexity while maintaining reliability
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
Enhances the ability of autonomous vehicles to safely navigate by accurately detecting hazards and road surface profiles, improving safety and comfort by enabling obstacle avoidance and adaptive vehicle control.
Implementation Method 1
LiDAR, which uses laser pulses to create detailed 3D representations of the environment
Implementation Method 2
LiDAR, which uses laser pulses
Implementation Method 3
RADAR, which uses radio waves to detect objects and measure distances
Data Source
AI summary
Embodiments relate to hazard detection in autonomous and semi-autonomous systems and applications. A transformer may use sampled image and LiDAR features to extract and decode a representation of one or more features of each point (e.g., refined height, range, driving condition, etc.) on a sampled surface (e.g., the road). These detections may be provided to one or more control components of an autonomous vehicle, which may use the detections to navigate, plan, or otherwise perform one or more operations. Some embodiments employ an automated approach to derive ground truth data from sensor data collected by data collection vehicle(s), such as data representing detected ground surface models, detected surface features, detected weather and/or surface condition labels, and/or detected per-point artifact labels. Accordingly, surface features such as ground surface heights along a predicted trajectory may be detected and ground truth data may be generated for a variety of sensing tasks.