Surround View Camera Calibration for Seamless 360° Image Stitching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing surround view systems face challenges in proper camera calibration, leading to non-aligned image stitching, ghosting, and faulty color correction due to occlusions and distortions from wide-angle lenses, which affect the seamless generation of a 360-degree composite view.
Innovation Solution
A method for calibrating multiple cameras using intrinsic and extrinsic parameters, including optical center, focal length, and lens distortion correction, followed by photometric alignment and seam identification to merge images, forming a virtual bird's-eye view.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Area of stationary object
If images from multiple cameras are stitched together to form a 360-degree view, then the coverage area is improved, but image alignment and seamlessness deteriorate due to calibration errors and wide-angle lens distortions
Solution Approach 1:
The system performs preliminary camera calibration before image stitching by detecting corners in a calibration arrangement and estimating extrinsic camera parameters. This preliminary action establishes accurate spatial relationships between cameras, preventing alignment issues during the stitching process and eliminating the need for post-processing corrections.
Solution Approach 2:
The system incrementally tunes corner location estimates based on pixels of projected images and adjusts extrinsic camera parameters accordingly. This dynamic parameter adjustment optimizes the calibration data to compensate for wide-angle lens distortions and ensures precise image alignment across the 360-degree view.
2Measurement precision
If corner detection is used to estimate extrinsic parameters, then calibration accuracy is improved, but detection precision deteriorates due to pixel-level uncertainties
Solution Approach 1:
The system projects detected corners onto image planes and uses the pixel values at these projected locations to incrementally tune the corner location estimates. This feedback loop continuously refines the corner positions and corresponding extrinsic parameters, converting initial detection uncertainties into precise calibrated values.
3Area of stationary object
If multiple cameras are used to capture surrounding views, then the field of view is improved, but system complexity increases due to calibration and image merging requirements
Solution Approach 1:
The system performs self-calibration by automatically detecting corners in a calibration arrangement and computing extrinsic parameters without requiring manual intervention. The corner detection and parameter estimation processes are automated, reducing the operational complexity despite using multiple cameras.
Solution Approach 2:
The system merges images from multiple cameras into a unified 360-degree view by establishing coordinate transformations based on calibrated extrinsic parameters. This merging process integrates data from multiple sources into a coherent single view, managing the complexity of multi-camera operations through systematic image composition.
Data Source
AI summary
Aspects of the method, apparatus, non-transitory computer readable medium, and system include estimating extrinsic camera parameters of a plurality of cameras based on detection of corners in a camera calibration arrangement by projecting an image of the detected corners onto an image plane of at least one of the plurality of cameras. The aspects further include incrementally tuning a location estimate for the detected corners based on pixels of the projected image, and adjusting the extrinsic camera parameters of the at least one of the plurality of cameras based on the tuned location estimate. The aspects further include generating a virtual image based on the pixels of the projected image mapped to a virtual image plane.