Surround View Camera Calibration for Seamless 360° Stitching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing surround view systems face challenges in properly calibrating multiple cameras to generate a seamless 360-degree composite view due to issues like non-aligned image stitching, ghosting, and faulty color correction, particularly with wide-angle cameras, which can lead to distortions and occlusions.
Innovation Solution
A method for calibrating multiple cameras using intrinsic and extrinsic parameters, including optical center, focal length, and lens distortion, to project images onto a virtual camera, and aligning images based on depth and photometric differences to create a seamless 360-degree composite view.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Area of stationary object
If images from multiple cameras are stitched together to form a 360-degree view, then a complete surround view is achieved, but image alignment errors cause non-aligned stitching, ghosting, and visual seams
Solution Approach 1:
The system performs preliminary calibration of extrinsic camera parameters and pre-computation of seam locations before generating the final surround view. By calculating optimal seam positions in advance based on camera geometry and depth information, the system prevents alignment errors rather than correcting them during stitching.
Solution Approach 2:
The system introduces an intermediary calibration process that uses detected corners and projected images to compute extrinsic parameters. This intermediary step acts as a mediator between raw camera images and the final stitched output, ensuring precise alignment through intermediate coordinate transformations and depth-based adjustments.
2Area of stationary object
If wide-angle cameras are used to capture broader views, then the field of view is increased, but lens distortion and occlusions increase
Solution Approach 1:
The system dynamically adjusts calibration parameters including extrinsic camera parameters, seam locations, and blending weights based on detected depth information and object proximity. By changing these parameters adaptively, the system compensates for wide-angle lens distortion and minimizes occlusion artifacts in the final composite image.
Solution Approach 2:
The system applies different processing qualities to different regions of the image. In overlap regions between adjacent camera views, it performs localized blending and seam adjustment based on depth information. This local quality approach allows aggressive distortion correction in critical areas while maintaining natural appearance in non-critical regions.
3Ease of manufacture
If multiple cameras are calibrated using traditional methods, then the calibration process is completed, but artifacts and visual seams remain in the composite image
Solution Approach 1:
The system uses depth information from detected objects as feedback to iteratively optimize seam locations and blending parameters. By continuously adjusting the calibration based on feedback from depth analysis and artifact detection, the system eliminates visual seams and ghosting that would persist with traditional fixed calibration methods.
Solution Approach 2:
The calibration system transitions from static traditional methods to dynamic adaptive calibration. Seam locations and blending weights are not fixed but dynamically adjusted based on real-time depth information, object detection results, and detected artifacts, allowing the system to optimize visual seamlessness for each specific scene configuration.
Data Source
AI summary
Aspects of the method, apparatus, non-transitory computer readable medium, and system include estimating extrinsic camera parameters of a plurality of cameras based on detection of corners in a camera calibration arrangement by projecting an image of the detected corners onto an image plane of at least one of the plurality of cameras. The aspects further include incrementally tuning a location estimate for the detected corners based on pixels of the projected image, and adjusting the extrinsic camera parameters of the at least one of the plurality of cameras based on the tuned location estimate. The aspects further include generating a virtual image based on the pixels of the projected image mapped to a virtual image plane.