Camera-Based Sensor Fusion for Autonomous Driving Perception
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Autonomous driving solutions face challenges with high costs and aesthetic limitations due to the use of LIDAR sensors, which are expensive and often mounted on vehicles' roofs, limiting their design flexibility.
Innovation Solution
A combination of cameras, including multipurpose time-of-flight (TOF) and RGB-IR cameras, along with other sensors like Radar and GPS, are used to perceive roadway conditions by fusing sensor data into a unified view, allowing for reliable autonomous driving without the need for expensive LIDAR sensors.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If LIDAR sensors are used for autonomous driving, then measurement precision and reliability are improved, but cost and device complexity increase
Solution Approach 1:
The patent combines multiple camera types (TOF and RGB-IR) with Radar and GPS sensors into a unified sensor fusion system. This merging approach replaces the need for expensive LIDAR sensors while achieving comparable environment perception accuracy through data integration from multiple sources.
Solution Approach 2:
The TOF camera is designed to operate in multiple modes: it can function as a standard camera for visual detection or switch to LIDAR mode for depth sensing and object tracking. This multi-functionality allows the system to achieve LIDAR-level measurement precision using a more cost-effective and less complex camera-based solution.
2Area of stationary object
If LIDAR sensors are mounted on the roof, then sensing coverage is improved, but aesthetic design flexibility deteriorates
Solution Approach 1:
Instead of mounting a single LIDAR sensor on the roof, the patent segments the sensing function across multiple camera sensors positioned at different locations on the vehicle body. This segmentation allows sensors to be integrated into the vehicle's existing structure without compromising aesthetic design, while collectively achieving comprehensive sensing coverage through sensor fusion.
3Device complexity
If camera-based systems are used instead of LIDAR, then cost and aesthetics are improved, but measurement precision in low-light conditions deteriorates
Solution Approach 1:
The patent employs a composite sensor system combining TOF cameras with infrared capabilities and RGB-IR cameras. This composite approach integrates multiple sensing modalities that complement each other: TOF provides depth information independent of light conditions, while RGB-IR cameras capture visual data across different spectral ranges, together achieving reliable object detection in low-light environments at lower cost than LIDAR.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This approach provides cost-effective and aesthetically flexible autonomous driving capabilities by using camera-based systems to process and fuse sensor data, enabling accurate object detection and vehicle control, even in low-light environments.
Implementation Method 1
a multipurpose time-of-flight (TOF) camera that can switch between a camera mode and a LIDAR mode
Implementation Method 2
one or more RGB-IR cameras. The one or more RGB-IR cameras use a wider FOV to sense objects closer to the vehicle
Data Source
AI summary
The present invention extends to methods, systems, and computer program products for perceiving roadway conditions from fused sensor data. Aspects of the invention use a combination of different types of cameras mounted to a vehicle to achieve visual perception for autonomous driving of the vehicle. Each camera in the combination of cameras generates sensor data by sensing at least part of the environment around the vehicle. The sensor data form each camera is fused together into a view of the environment around the vehicle. Sensor data from each camera (and, when appropriate, each other type of sensor) is fed into a central sensor perception chip. The central sensor perception chip uses a sensor fusion algorithm to fuse the sensor data into a view of the environment around the vehicle.