Vehicle Surround Visualization With Depth-Based 3D Object Rendering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing vehicle surround view systems suffer from visual artifacts such as geometric, texture, and color distortions, which obscure or omit useful information and can be distracting, interfering with safe vehicle operation.
Innovation Solution
The environment surrounding a vehicle is visualized by texturizing a detected 3D surface topology, with dynamic objects rendered by warping rigid objects and representing non-rigid objects as flat 2D surfaces, using depth maps and neural networks to estimate and model the environment's geometry.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Shape
If 2D images are projected onto a 3D bowl shape to create surround view visualization, then a three-dimensional visual representation is achieved, but geometric distortions and texture artifacts are introduced
Solution Approach 1:
The patent transitions from projecting 2D images onto a simplified 3D bowl shape to reconstructing the actual 3D geometry of the environment using depth maps and point clouds. This dimensional transformation from virtual projection to real geometry reconstruction eliminates geometric distortions while maintaining the 3D visualization benefit.
Solution Approach 2:
The patent replaces the mechanical projection system (projecting 2D images onto a fixed 3D bowl) with a computational geometry reconstruction system using neural networks, depth estimation, and point cloud processing. This substitution allows for accurate 3D representation without the artifacts inherent in fixed geometric projections.
2Area of stationary object
If camera frames are stitched together using blending techniques to provide 360° surround view, then a complete surrounding visualization is achieved, but texture distortions and color artifacts are introduced
Solution Approach 1:
The patent segments the environment into multiple depth maps corresponding to different camera views, processes each segment independently through neural network-based depth estimation, and then integrates them into a unified point cloud representation. This segmentation approach allows for precise texture mapping without the blending artifacts that occur when stitching images together.
Solution Approach 2:
The patent introduces point clouds as an intermediary representation between camera images and the final 3D visualization. Instead of directly blending 2D images, the system converts images to depth maps, reconstructs 3D points, and then renders from the point cloud data. This intermediary step preserves texture accuracy while achieving complete 360° coverage.
3Loss of information
If projection and stitching processes are used to create surround view visualization, then a virtual 3D representation is generated, but visual artifacts obscure useful information
Solution Approach 1:
The patent converts the harmful effect of geometric simplification (using a fixed 3D bowl shape) into a benefit by using neural networks to learn and reconstruct the actual complex geometry of the environment. The system transforms the limitation of simplified models into an advantage by leveraging deep learning to capture真实 environmental structures, thereby preserving visual information without artifacts.
Solution Approach 2:
The patent changes the fundamental parameters of the visualization system from fixed geometric projections to dynamic, learned geometry representations. By using neural networks to estimate depth maps and reconstruct point clouds, the system adapts the geometric parameters to match the actual environment, eliminating artifacts caused by mismatched projection geometries and preserving useful visual information.
Data Source
AI summary
In various examples, systems and methods are disclosed relating to geometry estimation and dynamic object rendering for vehicle environment visualization. In embodiments, the environment surrounding an ego-machine may be visualized by extracting one or more depth maps from image data, converting the depth map(s) into a 3D surface topology of the surrounding environment, and/or texturizing the detected 3D surface topology with image data. Dynamic objects such as moving vehicles or pedestrians may be detected and masked from a first pass of texturization. Rigid dynamic objects may be visualized by warping corresponding depth values using corresponding trajectories, inserting or fusing the resulting warped 3D representation of each such object into the (e.g., texturized) 3D surface topology, and texturizing the warped 3D representation of each object using corresponding image data. Non-rigid dynamic objects may be represented as flat 2D surfaces and texturized with corresponding image data.