Fisheye Lens Distortion Correction via 3D Mapping
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Omnidirectional cameras capture 360-degree video with fisheye lenses, resulting in distorted 2-dimensional images that need to be mapped to a 3-dimensional representation for accurate 360-degree video presentation, as the images are warped and compressed, lacking realistic distortion correction.
Innovation Solution
A method and apparatus that process 2-dimensional video data from omnidirectional cameras by mapping pixels in a circular image region to corresponding locations on a 3-dimensional hemispherical representation using polynomial equations, adjusting radial values for non-linear warping, and incorporating scaling parameters to correct distortion, enabling accurate 360-degree video presentation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Area of stationary object
If fisheye lenses are used to capture 360-degree video with omnidirectional cameras, then a wide field of view can be captured, but the resulting images exhibit severe distortion and warping
Solution Approach 1:
The patent transforms the distorted 2D fisheye image coordinates into a 3D spherical coordinate system, then maps them to corrected 2D output coordinates. This dimensional transformation allows the system to recover the true geometric relationships of the captured scene by introducing an intermediate 3D spherical representation that preserves angular information while eliminating radial distortion.
Solution Approach 2:
The patent applies parameter changes by using polynomial equations with adjustable coefficients to model the non-linear distortion characteristics of fisheye lenses. By fitting these polynomials to calibration data and adjusting the parameters during the coordinate transformation process, the system can accurately correct various types of fisheye distortion while maintaining the wide field of view.
2Quantity of substance
If 2-dimensional images are captured to represent 360-degree scenes, then data storage and transmission are simplified, but the scenes appear distorted and lack realistic representation
Solution Approach 1:
The patent introduces a virtual 3D spherical coordinate system as an intermediate representation layer between the captured 2D fisheye images and the final corrected output. This intermediate 3D representation preserves the angular and spatial relationships of the original scene while allowing for accurate mapping back to 2D display coordinates, thereby maintaining scene representation accuracy without increasing data volume.
3Measurement precision
If polynomial equations with adjusted radial values are used to map pixels, then distortion correction accuracy is improved, but computational complexity increases
Solution Approach 1:
The patent performs preliminary action by pre-calculating and storing polynomial coefficients through a calibration process using known reference patterns. These pre-computed parameters are then reused during real-time distortion correction, significantly reducing the computational burden during actual video processing while maintaining high correction accuracy. The heavy computational work is shifted to an offline calibration stage.
Data Source
Figure 1
Figure 2A~2B
Figure 3
AI summary
In various implementations, provided are systems and methods for correcting the distortion present in a fisheye image, and rendering the image for display as 360-degree video. In various implementations, a computing device can receive 2-dimensional video data captured by an omnidirectional camera. The computing device can map an image from each video frame to a 3-dimensional hemispherical representation. In various implementations, this mapping can be executed using a polynomial model. The 3-dimensional hemispherical representation can then be used in a 360-degree video presentation, to provide a virtual reality experience.