Surround View Image Stitching With Dynamic Seam Placement
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing vehicle Surround View Systems suffer from noticeable seams in stitched images due to noise and white balance variations, geometric and texture distortions, and the inability to visualize the area under the vehicle, leading to obscured useful visual information and potential safety hazards.
Innovation Solution
Implement dynamic seam placement based on object saliency, an adaptive 3D bowl model that changes shape based on distance and direction to detected objects, and real-time reconstruction of the area under the vehicle using cached sensor data and ego-motion, enhancing image stitching and visualization quality.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If conventional image stitching techniques are used to create surround view visualization, then a 360-degree view is achieved, but noticeable seams and stitching artifacts appear that obscure useful visual information
Solution Approach 1:
The patent applies local quality by making the seam placement dynamic and context-dependent rather than uniform. The system detects objects in the scene and adjusts seam locations to avoid placing them on important objects, thereby preserving visual information quality in critical regions while maintaining stitching functionality in less important areas.
Solution Approach 2:
The patent implements dynamics by making the seam placement adaptive and changeable based on real-time scene content. The seam locations are not fixed but are dynamically adjusted according to detected objects and their importance, allowing the stitching system to respond to varying visual conditions and minimize information loss in different scenarios.
2Loss of information
If ultrasonic sensors are used to detect objects for seam placement, then seams can be positioned to avoid close objects, but objects outside the ultrasonic sensing range are ignored
Solution Approach 1:
The patent merges multiple detection approaches by combining ultrasonic sensor data with computer vision-based object detection from camera images. This integration allows the system to leverage the short-range precision of ultrasonic sensors while also capturing objects at longer distances through image analysis, thereby extending the effective detection range and ensuring comprehensive seam placement decisions.
Solution Approach 2:
The patent uses image data as an intermediary to bridge the gap between ultrasonic sensor limitations and the need for comprehensive object detection. The image processing system acts as a mediator that detects objects beyond ultrasonic range and provides this information to the seam placement algorithm, enabling it to make informed decisions about seam locations for all detected objects regardless of distance.
3Loss of information
If multiple cameras capture images of moving objects, then a comprehensive surround view is achieved, but geometric and texture distortions occur due to parallax and different perspectives
Solution Approach 1:
The patent applies preliminary action by performing image stabilization and distortion correction on individual camera images before the stitching process. This preprocessing step compensates for camera motion and geometric distortions in advance, ensuring that images from multiple cameras can be aligned more accurately and reducing artifacts in the final stitched output.
Solution Approach 2:
The patent implements parameter changes by dynamically adjusting alignment and blending parameters based on detected object motion and camera state. The system modifies stitching parameters such as seam location, blending weights, and geometric transformation parameters to account for parallax and perspective differences, thereby minimizing distortions while maintaining accurate object representation.
4Loss of information
If a static 3D bowl model is used to represent the surrounding environment, then the visualization structure is simple, but the area under the vehicle remains invisible and depth accuracy is reduced
Solution Approach 1:
The patent applies dimensionality change by transitioning from a traditional 2D planar stitching approach to a 3D cylindrical or spherical coordinate system for image mapping. This dimensional transformation allows the system to represent the surround view in a three-dimensional space that naturally accommodates the area under the vehicle and provides more accurate depth perception while maintaining a manageable model structure.
Data Source
AI summary
In various examples, a state machine is used to select between a default seam placement or dynamic seam placement that avoids salient regions, and to enable and disable dynamic seam placement based on speed of ego-motion, direction of ego-motion, proximity to salient objects, active viewport, driver gaze, and/or other factors. Images representing overlapping views of an environment may be aligned to create an aligned composite image or surface (e.g., a panorama, a 360° image, bowl shaped surface) with overlapping regions of image data, and a default or dynamic seam placement may be selected based on driving scenario (e.g., driving direction, speed, proximity to nearby objects). As such, seams may be positioned in the overlapping regions of image data, and the image data may be blended at the seams to create a stitched image or surface (e.g., a stitched panorama, stitched 360° image, stitched textured surface).