3D Stixel Representation for Autonomous Surroundings
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for generating 3D representations of surroundings for autonomous driving and robotics are complex and resource-intensive, requiring extensive computing resources and post-processing, making them unsuitable for embedded devices and inadequate for full self-driving scenarios.
Innovation Solution
An end-to-end CNN-based approach that generates multiple 3D stixels for each image column, reducing the need for semantic segmentation and depth estimation, and enabling efficient representation of complete 3D surroundings using a convolutional neural network trained to parameterize stixels with depth and semantic information.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If deep learning-based algorithms are used for semantic segmentation and depth calculation, then measurement precision is improved, but device complexity increases
Solution Approach 1:
The patent segments the continuous depth space into discrete depth bins and divides the image into columns, processing each column independently to generate stixels. This segmentation approach reduces computational complexity by breaking down the complex semantic segmentation and depth estimation problem into simpler, independent column-wise processing tasks while maintaining depth measurement precision through the stixel representation.
Solution Approach 2:
The patent transforms the continuous depth values into discrete depth bins and represents the 3D environment using stixel parameters (position, depth, height) instead of full semantic segmentation maps. This parameter transformation reduces the computational burden by working with simplified representations while preserving essential depth and spatial information needed for autonomous navigation.
2Measurement precision
If conventional semantic segmentation and depth estimation approaches are used, then measurement precision is improved, but productivity decreases
Solution Approach 1:
The patent performs preliminary column-wise processing of the image, generating stixels for each column independently before assembling the complete environment representation. This preliminary action in each column allows parallel processing and reduces the overall processing time while maintaining accurate depth and spatial measurements through the stixel framework.
Solution Approach 2:
The patent transitions from 2D image processing to 3D stixel representation by adding the depth dimension through discrete depth bins. This dimensional transformation enables efficient processing by representing the environment in a compact 3D format that captures essential spatial information while reducing computational requirements compared to full semantic segmentation.
3Adaptability or versatility
If slanted stixels are used for 3D representation, then adaptability is improved, but device complexity increases
Solution Approach 1:
The patent uses dynamic fusion algorithms including Kalman filters to track and update stixels over time, allowing the system to adapt to changing environments and maintain accurate 3D representations. This dynamic approach enables the system to handle moving objects and changing scenes while managing computational requirements through efficient recursive updates rather than complete reprocessing.
Solution Approach 2:
The patent introduces stixels as an intermediary representation between the 2D camera image and the 3D environment model. These stixels serve as a computational mediator that simplifies the transformation process, enabling efficient 3D representation and fusion algorithms while reducing the complexity of direct 3D reconstruction from 2D images.
Data Source
AI summary
The invention relates to a method (100) for generating a representation (70) of the surroundings, comprising the following steps:providing (101) at least one image (30) that results from a recording by an image detection device (5) and that represents objects (6) and/or surfaces (6) in the surroundings (7) of the image detection device (5), wherein the provided image (30) is subdivided into multiple image columns (31),generating (102) the representation (70) of the surroundings, wherein for this purpose multiple three-dimensional stixels (80) for each image column (31) of the provided image (30) are parameterized for representing the objects (6) and/or surfaces (6) in three-dimensional space,wherein the generation (102) of the representation (70) of the surroundings takes place using a model (50) which uses the provided image (30) as input.