Fisheye Camera 3D Projection for Undistorted Surround View
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Combining images from multiple fisheye cameras with different tilt angles and wide baselines poses challenges due to distortion and alignment issues, particularly in advanced driver assistance systems where a clear 360-degree surround view is required, as existing methods like panorama stitching fail to effectively handle large FOVs and distortions.
Innovation Solution
The system models each fisheye camera as a 3D shape, projects images onto a common 3D shape, and then aligns and stitches them onto a common plane to generate a combined view, compensating for tilt and rotation to minimize alignment errors and preserve image content.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Area of moving object
If wide-angle lenses are employed to capture large FOV images, then the field of view is improved, but image distortion increases
Solution Approach 1:
The patent transforms 2D fisheye images into 3D spherical representations, then projects them onto a common plane. This dimensional transformation allows the system to preserve the wide field of view while correcting distortion through geometric projection mathematics, effectively resolving the contradiction between large FOV and image accuracy
Solution Approach 2:
The system changes the projection parameters by modeling each camera's view as a 3D shape and transforming coordinates from spherical to planar space. This parameter transformation enables distortion correction while maintaining the expansive field of view captured by wide-angle lenses
2Area of stationary object
If images from multiple cameras with different tilt angles are combined, then the coverage area is improved, but alignment accuracy deteriorates
Solution Approach 1:
By elevating 2D images to 3D spherical models, the system creates a common geometric framework that naturally accommodates different camera tilt angles. The 3D-to-2D projection process then automatically aligns all views to a common plane, resolving alignment issues while preserving comprehensive coverage
Solution Approach 2:
The patent introduces 3D spherical representations as an intermediary between the raw fisheye images and the final combined view. This intermediate 3D model serves as a common reference frame that facilitates accurate alignment of images from cameras with different orientations, thereby maintaining both coverage and precision
3Ease of manufacture
If traditional panorama stitching is used for wide-angle images, then the processing simplicity is improved, but the effectiveness in handling large FOV and distortion deteriorates
Solution Approach 1:
The patent replaces traditional 2D panorama stitching with a 3D spherical projection approach. This dimensional elevation provides a mathematically rigorous framework for handling wide-angle distortion, achieving both effectiveness in distortion correction and reasonable processing complexity through systematic coordinate transformations
Data Source
Figure 1
Figure 2
Figure 3
AI summary
An electronic device is described. The electronic device includes a processor. The processor is configured to obtain images from a plurality of cameras. The processor is also configured to project each image to a respective 3-dimensional (3D) shape for each camera. The processor is further configured to generate a combined view from the images.