Top-View 3D Stixel Fusion for Vehicle Obstacle Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing obstacle detection systems in vehicles lack a comprehensive method to accurately represent free space in a three-dimensional format, failing to effectively integrate multi-sensor data for autonomous or semi-autonomous navigation.
Innovation Solution
A sensor fusion-based top-view three-dimensional stixel representation system that combines data from cameras, lidar, and radar systems using neural networks to create a unified feature representation in a polar coordinate system, encoding attributes like distance, height, type, appearance, and color, facilitating general obstacle detection.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If multiple sensors (camera, lidar, radar) are integrated to obtain comprehensive environmental data, then the completeness and accuracy of obstacle detection is improved, but the device complexity and data processing requirements increase
Solution Approach 1:
The patent combines data from multiple sensor types (camera, lidar, radar) into a unified sensor fusion system. The processing circuitry integrates heterogeneous sensor data streams to create a comprehensive environmental representation, merging the strengths of each sensor modality to achieve accurate 3D stixel representations that no single sensor could provide alone.
Solution Approach 2:
The sensor fusion system is designed to handle multiple sensor types and data formats through a universal processing architecture. The neural network and processing circuitry are configured to accept and process data from various sensor sources (2D images from cameras, 3D point clouds from lidar, radar data) and transform them into a common top-view 3D stixel representation format, enabling multi-functional operation across different sensor configurations.
2Loss of information
If a three-dimensional top-view stixel representation is created from multi-sensor data, then the ability to represent free space and objects comprehensively is improved, but the computational processing time and complexity increase
Solution Approach 1:
The system performs preliminary transformations of sensor data into top-view representations before fusion. Each sensor's data is pre-processed and projected into a top-view coordinate system in advance, which simplifies the subsequent fusion process and enables faster generation of the final 3D stixel representation compared to fusing raw sensor data and then transforming.
Solution Approach 2:
The patent transforms sensor data from various coordinate systems and dimensions into a unified top-view 3D representation. Camera 2D images are projected to top-view, lidar 3D point clouds are transformed to top-view coordinates, and all sensor data is fused in this intermediate top-view dimension before being converted to the final polar coordinate 3D stixel representation, enabling efficient multi-dimensional data integration.
3Adaptability or versatility
If sensor data is transformed to a polar coordinate system for stixel representation, then the suitability for autonomous navigation and obstacle detection is improved, but the coordinate transformation complexity increases
Solution Approach 1:
The system changes the coordinate system parameters from Cartesian (used by most sensors) to polar coordinates for the final stixel representation. This parameter transformation is performed through the processing circuitry and neural network, converting the fused top-view data into polar coordinates where the origin is at the vehicle position, enabling direct mapping to stixel lengths and angles that are naturally suited for navigation decisions.
Data Source
AI summary
A system in a vehicle includes a first sensor to obtain first sensor data from a first field of view and provide a first top-view feature representation. The system also includes a second sensor to obtain second sensor data from a second field of view with an overlap with the first field of view and provide a second top-view feature representation. Processing circuitry implements a neural network and provides a top-view stixel representation based on the first top-view feature representation and the second top-view feature representation. The top-view three-dimensional stixel representation is used to control an operation of the vehicle.