Polarized Image Processing for Accurate Normal Line Extraction

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Existing image processing methods fail to accurately separate and account for reflection components in polarized images, leading to inaccuracies in generating normal line information.

Innovation Solution

An image processing apparatus and method that utilizes an image pickup element with identical polarization pixel blocks and color filters to generate polarized images with high extinction ratios, allowing for the separation and extraction of reflection components, which are then used to generate highly accurate normal line information.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If conventional image processing methods are used to generate normal line information from polarized images, then the processing is simpler, but the accuracy of normal line information is insufficient due to inability to separate reflection components

Engineering Contradiction:
Improveaccuracy of normal line informationVSAvoidcomplexity of image processing
Core Design Contradiction:
Measurement precisionVSDevice complexity

Solution Approach 1:

The invention segments the polarized image into multiple reflection components (specular reflection, diffuse reflection, and substrate reflection) based on their different polarization characteristics. By dividing the image processing into component-specific processing steps, the system achieves high measurement precision for normal line information while managing complexity through systematic segmentation of the reflection types.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The invention introduces polarization degree as an intermediary parameter to differentiate and separate various reflection components. By calculating and utilizing the polarization degree of each pixel, the system can identify and process different reflection types (specular, diffuse, substrate) separately, enabling accurate normal line information extraction without requiring complex direct separation methods.

Inventive Principle:
Principle #24Intermediary (Mediator)

2Measurement precision

If polarized images are captured without considering reflection components, then the imaging process is faster, but the generated normal line information contains inaccuracies

Engineering Contradiction:
Improveaccuracy of normal line informationVSAvoidprocessing speed
Core Design Contradiction:
Measurement precisionVSProductivity

Solution Approach 1:

The invention performs preliminary classification of pixels into different reflection component categories (specular, diffuse, substrate) based on their polarization characteristics before generating normal line information. This preliminary segmentation allows subsequent processing to be optimized for each component type, improving both accuracy and processing efficiency by avoiding unnecessary computations for each pixel.

Inventive Principle:
Principle #10Preliminary action

Solution Approach 2:

The invention utilizes changes in polarization degree as a key parameter to differentiate reflection components. By monitoring and utilizing the polarization degree parameter across different image channels, the system can rapidly classify and process different reflection types, maintaining high productivity while achieving accurate normal line information extraction.

Inventive Principle:
Principle #35Parameter changes

Applied Scientific Principles

This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.

Function Achieved in This Case

The solution enables the generation of highly accurate normal line information by effectively separating and accounting for reflection components, improving the accuracy of three-dimensional shape acquisition and object recognition.

Implementation Method 1

a polarized image with a plurality of polarization directions is generated using an image pickup element and a polarizer

Methodology Applied
Scientific EffectPolarization: Polarisation

Data Source

PatentEP3264754B1Image processing device, image processing method and image capturing element
Publication Date: 2022.02.16 SONY GROUP CORP
  • EP3264754B1 patent drawingFigure 1
  • EP3264754B1 patent drawingFigure 2
  • EP3264754B1 patent drawingFigure 3

AI summary

An imaging unit 20 has a configuration in which an identical polarization pixel block made up of a plurality of pixels with an identical polarization direction is provided for each of a plurality of polarization directions and pixels of respective predetermined colors are provided in the identical polarization pixel block. A correction processing unit 31 performs correction processing such as white balance correction on a polarized image generated by the imaging unit 20. A polarized image processing unit 32 separates or extracts a reflection component using the polarized image after the correction processing. By using a polarized image of the separated or extracted reflection component, for example, it is possible to generate normal line information with high accuracy.