Under-Vehicle Reconstruction for Seamless Surround View Visualization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing vehicle Surround View Systems suffer from noticeable seams in stitched images due to noise and white balance variations, geometric and texture distortions, and the inability to accurately represent the 3D environment, leading to artifacts like ghosting and object disappearance, and they lack visualization of the area under the vehicle.
Innovation Solution
Implement dynamic seam placement based on object saliency and ego-object state, an adaptive 3D bowl model that changes shape based on detected objects, and real-time reconstruction of the area under the vehicle, along with optimized visualization streaming for improved image stitching and environmental representation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional image stitching techniques are used to create surround view visualization, then a 360° view can be provided, but noticeable seams and stitching artifacts appear in the stitched image
Solution Approach 1:
The patent extracts and removes stitching seams from the final surround view visualization by using multiple overlapping images and selectively blending regions. The system identifies seam locations and excludes them from the composite image, thereby eliminating the harmful visual artifacts while maintaining the 360° view capability.
Solution Approach 2:
The patent combines multiple fisheye camera images with overlapping fields of view into a single composite surround view. By merging multiple images and using blending techniques in overlapping regions, the system eliminates seams and artifacts that would appear in single-image stitching, achieving a seamless 360° visualization.
2Measurement precision
If ultrasonic sensors are used to detect objects for seam placement, then close objects can be identified, but objects outside the short sensing range are ignored
Solution Approach 1:
The patent uses the surround view camera system for multiple purposes: both creating the visual display and detecting objects for seam placement. This multi-functional approach replaces ultrasonic sensors, enabling detection of objects at all distances within camera range, not just close objects, thereby preventing seams from occluding important distant objects.
Solution Approach 2:
The patent uses the already-captured image data as an intermediary to detect objects and determine seam placement. Instead of relying on ultrasonic sensors with limited range, the system analyzes the visual information from fisheye cameras to identify objects and position seams accordingly, extending detection capability to all visible distances.
3Area of stationary object
If fisheye cameras are used to capture surround views, then a wide field of view is achieved, but geometric and texture distortions occur in the stitched image
Solution Approach 1:
The patent divides the surround view into multiple overlapping images from different fisheye cameras rather than attempting to stitch a single distorted image. By segmenting the view and using multiple perspectives with overlapping regions, the system reduces geometric and texture distortions while maintaining a wide effective field of view.
4Device complexity
If the area under the vehicle is not included in the visualization, then the system complexity is reduced, but a comprehensive view of the environment is lost
Solution Approach 1:
The patent extends the surround view visualization into a third dimension by including the area under the vehicle. This is achieved by projecting or rendering the under-vehicle region onto the existing 2D display, creating a multi-dimensional visualization that provides comprehensive environmental awareness without requiring additional physical sensors under the vehicle.
Data Source
AI summary
In various examples, cached sensor data captured by an ego-object and ego-motion of the ego-object are used to reconstruct the area under the vehicle in real time. For example, image data captured over time by a vehicle may be cached into a composite map that visualizes the ground or drivable area, and the vehicle's ego-motion may be used to retrieve a region of the composite map corresponding to the under vehicle area. For each time slice, a newly captured or generated image representing that time slice may be used to generate a local map of an observed portion of the ground, and the local map may be merged with a composite map that represents previously observed local maps. Accordingly, the under vehicle area for that time slice may be reconstructed by retrieving corresponding pixels from the composite map using the vehicle's ego-motion.