3D Obstacle Detection Using Virtual Stixel Models
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional methods for object and free space detection in robotic systems, particularly for autonomous driving, are complex and expensive, relying on multi-sensor systems that are difficult to synchronize and require extensive engineering, while classical computer vision techniques are not effective end-to-end.
Innovation Solution
A method involving ground truth generation, obstacle and free space detection, and generalization across different cameras using machine-learning algorithms, specifically convolutional neural networks, to enable precise object detection and 3D obstacle stixel prediction from single-image representations, reducing the need for expensive 3D sensors.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If multi-sensor systems (LiDAR, radar) are used for 3D obstacle detection, then measurement precision is improved, but device complexity and cost increase
Solution Approach 1:
The patent creates a virtual 3D point cloud representation (stixel model) as a copy of the real-world environment from 2D camera images. This virtual copy contains all necessary 3D spatial information without requiring physical 3D sensors, thus achieving accurate obstacle detection while avoiding multi-sensor system complexity
Solution Approach 2:
The patent replaces the mechanical/optical 3D sensing system (LiDAR, radar) with a computational approach using 2D camera images and machine learning algorithms. The physical sensor system is substituted by an information processing system that reconstructs 3D space computationally
2Measurement precision
If multi-sensor systems are used for obstacle detection, then measurement precision is improved, but ease of operation deteriorates due to synchronization requirements
Solution Approach 1:
The method creates a unified virtual 3D representation from 2D images that inherently contains synchronized spatial information. By working with a computational copy of the environment rather than fusing multiple sensor streams, the synchronization problem is eliminated while maintaining detection accuracy
3Ease of manufacture
If classical computer vision techniques are used for object detection, then ease of manufacture is improved, but productivity and detection effectiveness deteriorate
Solution Approach 1:
The patent replaces traditional multi-stage computer vision processing with an end-to-end deep learning system. The classical pipeline of separate detection, segmentation, and 3D reconstruction steps is substituted by a unified neural network that performs all tasks simultaneously, improving both efficiency and effectiveness
4Measurement precision
If expensive 3D sensors are used for autonomous driving, then measurement precision is improved, but loss of substance (cost) increases
Solution Approach 1:
The patent creates a virtual 3D point cloud (stixel model) as a computational copy of the physical environment from inexpensive 2D camera images. This virtual copy provides accurate 3D spatial information without requiring expensive physical 3D sensors, eliminating the cost overhead while maintaining measurement precision
Solution Approach 2:
The method uses inexpensive 2D camera images as the primary sensing modality instead of expensive 3D sensors. The computational 3D reconstruction serves as a disposable virtual representation that can be regenerated from cheap image data, avoiding the need for costly hardware
Data Source
AI summary
A method for detecting information about at least one object and/or at least a part of the free space in a representation (4) of the environment of a system. The method comprises: a) performing ground truth generation; b) performing an obstacle and clearance detection; c) performing a generalization across different cameras and/or different digital image representations.