Pseudo 3D Image Generation via Sky Polarization Normal Estimation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional methods for generating pseudo 3D images from 2D images outdoors face challenges, particularly when estimating surface normals from distant objects like huge buildings, as they require specialized devices or inaccurate assumptions, and are limited by weather conditions such as sunlight and cloud cover.
Innovation Solution
An image processor that captures polarization images, determines weather conditions, and estimates surface normals using a whole sky polarization map, allowing for accurate normal estimation regardless of weather by detecting the polarization state of specular reflected light and selecting appropriate methods based on weather conditions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional methods (laser rangefinder or stereoscopic shooting) are used to extract surface normal from 2D image, then surface normal information can be obtained, but these methods are not practicable for distant objects and require specialized equipment
Solution Approach 1:
The patent replaces mechanical/optical measurement systems (laser rangefinder, stereoscopic cameras) with a computational approach using polarization image processing. By analyzing the polarization state of specular reflected light in 2D images and comparing it with pre-acquired whole sky polarization maps, the system estimates surface normals without requiring specialized hardware or complex mechanical setups.
Solution Approach 2:
The patent introduces whole sky polarization maps as an intermediary reference dataset. These pre-acquired polarization patterns serve as a mediator between the observed polarization state in the 2D image and the underlying surface normal information, enabling indirect estimation without direct measurement.
2Measurement precision
If polarization information is used to estimate surface normal outdoors, then accurate normal estimation can be achieved, but the method is affected by weather conditions such as sunlight and cloud cover
Solution Approach 1:
The patent performs preliminary acquisition of whole sky polarization maps under various weather conditions before actual surface normal estimation. These pre-acquired reference maps capture the polarization patterns of the sky under different lighting conditions, allowing the system to adapt to varying weather situations by selecting appropriate reference data.
Solution Approach 2:
The patent changes the reference polarization map parameters based on detected weather conditions. By comparing current sky polarization characteristics with stored reference maps, the system selects or adjusts the appropriate polarization map parameters to match current atmospheric conditions, thereby maintaining estimation accuracy across varying weather scenarios.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
Enables the generation of pseudo 3D images from 2D images by accurately estimating surface normals even for distant objects, overcoming limitations of conventional methods and providing effective image processing for various outdoor conditions.
Implementation Method 1
a polarization image capturing section for capturing a polarization image that provides polarization information of multiple pixels
Implementation Method 2
detecting the polarization state of specular reflected light at the object's surface
Data Source
AI summary
The present invention provides an apparatus that includes a color and polarization image capturing section 201, a whole sky polarization map getting section 202, a weather determining section 203, a fine-weather sky part separating section 204, a cloudy-weather normal estimating section 207, and a pseudo 3D image generating section 208. The apparatus obtains polarization information outdoors with the polarization state of the sky taken into account, and estimates surface normal information at an object's surface on a two-dimensional image, thereby generating a surface normal image. Using that normal image, the apparatus divides the object into multiple ranges, extracts three-dimensional information and generates a viewpoint changed image, thereby generating a pseudo 3D image.