Adaptive 3D Bowl Model for Surround View Stitching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing vehicle Surround View Systems suffer from noticeable seams in stitched images due to noise and white balance variations, geometric and texture distortions, and the inability to visualize the area under the vehicle, leading to obscured useful visual information and potential safety hazards.
Innovation Solution
Implement dynamic seam placement based on object saliency and ego-object state, adaptive 3D bowl modeling that adjusts shape based on distance and direction to detected objects, and real-time reconstruction of the area under the vehicle using cached sensor data and ego-motion.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Area of stationary object
If conventional image stitching techniques are used to create surround view visualization, then the system can provide a 360-degree view of the surrounding environment, but noticeable seams and stitching artifacts appear in the stitched images
Solution Approach 1:
The patent divides the stitching process into multiple stages: identifying salient objects in overlapping regions, determining optimal seam locations that avoid these objects, and applying localized blending techniques. This segmentation allows the system to treat different regions of the stitched image differently, preserving important visual information while minimizing visible seams.
Solution Approach 2:
The patent applies different processing strategies to different regions of the stitched image. In regions containing salient objects, the system prioritizes object preservation and uses adaptive blending. In regions without important objects, conventional stitching techniques are sufficient. This local quality approach ensures that stitching artifacts are minimized where they would be most noticeable.
2Reliability
If ultrasonic sensors are used to detect objects for seam placement, then seams can be positioned to avoid very close objects, but objects outside the ultrasonic sensing range are ignored and seams may be placed over important regions
Solution Approach 1:
The patent combines data from multiple sensor types with different detection ranges and characteristics. Ultrasonic sensors provide accurate detection of very close objects, while other sensors detect objects at longer distances. By merging these data sources, the system achieves both the reliability of close-object detection and the extended range needed to identify all salient objects for optimal seam placement.
Solution Approach 2:
The patent introduces an intermediary processing layer that receives data from ultrasonic sensors and other sensors, identifies salient objects across all detection ranges, and uses this comprehensive information to determine optimal seam locations. This intermediary layer reconciles the limited range of ultrasonic sensors with the need for broad detection coverage.
3Area of stationary object
If multiple cameras capture images of moving objects from different perspectives, then complete surround view coverage is achieved, but geometric distortions and ghosting effects occur in the stitched image
Solution Approach 1:
The patent implements dynamic adjustment of stitching parameters based on detected object motion. When moving objects are detected in overlapping regions, the system adapts the alignment and blending parameters in real-time to account for the object's movement between camera captures. This dynamic approach prevents ghosting effects that would occur with static stitching parameters.
Solution Approach 2:
The patent performs preliminary identification of moving objects in overlapping regions before finalizing the stitching process. By detecting motion in advance and adjusting the stitching strategy accordingly, the system prevents geometric distortions and ghosting effects rather than attempting to correct them after stitching.
4Shape
If a static 3D bowl model is used to represent the surrounding environment, then the system can provide a three-dimensional visualization, but the area under the vehicle cannot be visualized and useful visual information is obscured
Solution Approach 1:
The patent extends the traditional 3D bowl model by adding capabilities to represent and visualize the area under the vehicle. This dimensionality extension allows the system to incorporate data from cameras mounted on the vehicle body and reconstruct the ground plane and objects beneath the vehicle, providing a complete volumetric representation of the surrounding environment including previously hidden regions.
Data Source
AI summary
In various examples, an environment surrounding an ego-object is visualized using an adaptive 3D bowl that models the environment with a shape that changes based on distance (and direction) to one or more representative point(s) on detected objects. Distance (and direction) to detected objects may be determined using 3D object detection or a top-down 2D or 3D occupancy grid, and used to adapt the shape of the adaptive 3D bowl in various ways (e.g., by sizing its ground plane to fit within the distance to the closest detected object, fitting a shape using an optimization algorithm). The adaptive 3D bowl may be enabled or disabled during each time slice (e.g., based on ego-speed), and the 3D bowl for each time slice may be used to render a visualization of the environment (e.g., a top-down projection image, a textured 3D bowl, and/or a rendered view thereof).