Polarized Image Normal Line Calculation for Object Recognition

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Existing image processing technologies face challenges in accurately calculating normal lines from polarized images due to 180-degree periodicity issues and inability to identify three-dimensional shapes using only two polarization directions, and require high distance resolution images which are difficult to acquire.

Innovation Solution

An image processing apparatus and method that acquires polarized images with multiple polarization directions, calculates normal lines, and resolves uncertainties using temporary recognition processes and pre-registered models to determine object shapes and orientations accurately.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If normal lines are calculated from polarized images with multiple polarization directions, then the brightness information is improved, but 180-degree uncertainty remains in the azimuth angle determination

Engineering Contradiction:
Improvebrightness measurement precisionVSAvoidazimuth angle information
Core Design Contradiction:
Measurement precisionVSLoss of information

Solution Approach 1:

The patent introduces a temporary recognition process as an intermediary step. This process generates a temporary recognition process image from the polarized images and uses it to temporarily recognize the object, which then serves as a mediator to resolve the 180-degree uncertainty in the normal line calculation by providing contextual information about the object's identity and orientation

Inventive Principle:
Principle #24Intermediary (Mediator)

Solution Approach 2:

The patent performs preliminary object recognition using a temporary recognition process before finalizing the normal line calculation. This preliminary action of temporarily recognizing the object based on the temporary recognition process image allows the system to prepare the necessary information (such as expected object orientation and identity) that will be used to resolve the azimuth angle uncertainty in the subsequent normal line calculation step

Inventive Principle:
Principle #10Preliminary action

2Device complexity

If only two polarization directions are used, then the device complexity is reduced, but the ability to identify three-dimensional shape is insufficient

Engineering Contradiction:
Improvepolarization measurement system complexityVSAvoidthree-dimensional shape measurement precision
Core Design Contradiction:
Device complexityVSMeasurement precision

Solution Approach 1:

The patent transitions from using only intensity information to utilizing polarization direction information as an additional dimension. By calculating normal lines from the polarization characteristics of reflected light across multiple polarization directions, the system extracts three-dimensional shape information without requiring complex multi-camera or structured light systems, effectively adding a dimensional aspect to the measurement

Inventive Principle:
Principle #17Another dimension (Dimensionality change)

3Measurement precision

If high distance resolution images are used for accurate target detection, then the measurement precision is improved, but the difficulty of acquiring such images increases

Engineering Contradiction:
Improvedistance measurement precisionVSAvoidimage acquisition difficulty
Core Design Contradiction:
Measurement precisionVSDifficulty of detecting and measuring

Solution Approach 1:

The patent replaces mechanical or complex imaging systems (such as high-resolution depth cameras or structured light systems) with a polarization-based optical measurement approach. By utilizing the polarization characteristics of light reflected from the target object and applying physical models of polarization reflection, the system achieves accurate distance and shape measurement without requiring mechanically complex high-resolution imaging equipment

Inventive Principle:
Principle #28Mechanics substitution (Replace mechanical system)

Applied Scientific Principles

This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.

Function Achieved in This Case

Enables accurate recognition of objects with high precision by resolving normal line uncertainties and determining three-dimensional shapes from polarized images, improving upon existing limitations in object identification and shape determination.

Implementation Method 1

polarized images having a plurality of different polarization directions

Methodology Applied
Scientific EffectPolarization: Polarisation

Implementation Method 2

light reflected from the reference plane

Methodology Applied
Scientific EffectLight reflection: Reflection

Data Source

PatentEP3291052B1Image processing device, image processing method, and program
Publication Date: 2022.10.05 SONY GROUP CORP
  • EP3291052B1 patent drawingFigure 1
  • EP3291052B1 patent drawingFigure 2
  • EP3291052B1 patent drawingFigure 3

AI summary

A polarized image acquisition section 20 acquires a plurality of polarized images having different polarization directions. The polarized images show, for example, an input indicator for a user interface as a recognition target object. A normal line calculation section 30 calculates normal lines for individual pixels of the recognition target object in accordance with the polarized images acquired by the polarized image acquisition section 20. The normal lines represent information based on the three-dimensional shape of the recognition target object. A recognition section 40 recognizes the object by using the normal lines calculated by the normal line calculation section 30, determines, for example, the type, position, and posture of the input indicator, and outputs the result of determination as input information on the user interface. The object can be recognized easily and with high accuracy.