Visible-Infrared Image Blur Correction via Motion Estimation

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing image processing technologies struggle to effectively reduce blur in visible light images captured in dark environments, especially when camera motion is involved, as they require multiple continuously-captured images and are less effective for moving cameras.

Innovation Solution

An image processing device and method that utilizes both visible light and infrared images, along with camera motion information, to estimate and correct blur in visible light images by generating an integrated filter to remove or reduce blur, even in cases of moving cameras.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Manufacturing precision

If multiple continuously-captured images are used to analyze image motion and correct defocusing, then defocusing correction can be performed, but processing cannot be executed for stationary images and immediate processing for each image cannot be performed

Engineering Contradiction:

Improvedefocusing correction accuracyVSAvoidprocessing speed and immediate processing capability

Core Design Contradiction:

Manufacturing precisionVSProductivity

Solution Approach 1:

The patent uses an infrared image as a copy/reference of the visible light image to estimate blur characteristics. Instead of analyzing motion from multiple continuous frames, the system copies the spatial information from the infrared image to infer the blur pattern in the visible light image, enabling immediate processing without requiring multiple continuous captures.

Inventive Principle:

Principle #26Copying

Solution Approach 2:

The infrared image serves as an intermediary to bridge the visible light image and the blur estimation. By using the infrared image as a mediator, the system can estimate blur characteristics without directly analyzing motion from multiple visible light frames, thus enabling faster processing while maintaining correction accuracy.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Manufacturing precision

If defocusing correction is performed for fixed camera images, then blur can be reduced, but the effect is less exerted for moving cameras such as in-vehicle cameras

Engineering Contradiction:

Improvedefocusing correction accuracyVSAvoideffectiveness for moving cameras

Core Design Contradiction:

Manufacturing precisionVSAdaptability or versatility

Solution Approach 1:

The patent changes the approach from analyzing temporal motion patterns (suitable for fixed cameras) to using spatial information from infrared images (effective for moving cameras). By transforming the blur estimation basis from temporal to spatial parameters, the system adapts to moving camera scenarios while maintaining correction effectiveness.

Inventive Principle:

Principle #35Parameter changes

Solution Approach 2:

The patent replaces the mechanical motion analysis approach (tracking changes across multiple frames) with an optical/infrared-based estimation approach. By substituting the analysis method from visible light frame comparison to infrared image-based estimation, the system becomes effective for moving cameras where motion analysis is challenging.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

3Illumination intensity

If long exposure time is used to capture visible light images in dark environments, then image brightness is improved, but blur due to camera motion or object motion is easily generated

Engineering Contradiction:

Improveimage brightnessVSAvoidimage sharpness and focus

Core Design Contradiction:

Illumination intensityVSManufacturing precision

Solution Approach 1:

The patent converts the harmful effect of motion blur into a beneficial estimation opportunity. By using the infrared image (which captures the scene without motion blur) as a reference, the system can estimate and correct the blur in the visible light image, turning the limitation of long exposure into a solvable problem through cross-modality information utilization.

Inventive Principle:

Principle #22Blessing in disguise (Convert harm into benefit)

Solution Approach 2:

The infrared image acts as an intermediary that provides sharp reference information without suffering from motion blur. This intermediary allows the system to separate the brightness advantage of long exposure from the harmful blur effect, enabling correction by comparing the blurred visible light image with the sharp infrared reference.

Inventive Principle:

Principle #24Intermediary (Mediator)

Data Source

PatentUS11288777B2Image processing device, image processing method, and program

Publication Date: 2022.03.29 SONY GROUP CORP

AI summary

Image processing methods and apparatus are described. The image processing method comprises receiving input of a visible-ray image and an infrared-ray image obtained by photographing a same subject, estimating, based on the visible-ray image, the infrared-ray image and motion information, a blur estimate associated with the visible-ray image, and generating, based on the estimated blur estimate, a corrected visible-ray image.