Event Camera Image Restoration via Segmented Luminance Updates

Solution Overview

Problem

Existing image restoration techniques from event camera outputs face increased computational load when attempting to restore luminance images across the entire photographed surface, especially in areas without detectable changes.

Innovation Solution

An image restoration device and method that initializes luminance values to intermediate levels, updates these values based on pixel coordinates and polarity values from event cameras, and outputs binary images by overwriting firing coordinates with polarity values while retaining non-firing coordinates.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Area of stationary object

If simultaneous optical flow and luminance estimation is performed to restore luminance images across the entire photographed surface, then the luminance estimation coverage is improved, but the computational load increases

Engineering Contradiction:

Improveluminance estimation coverage areaVSAvoidcomputation process complexity

Core Design Contradiction:

Area of stationary objectVSDevice complexity

Solution Approach 1:

The patent divides the image restoration process into two distinct stages: edge detection phase (using optical flow for areas with luminance changes) and non-edge area restoration phase (using temporal integration of event data). This segmentation allows different processing methods to be applied to different regions, reducing overall computational complexity while maintaining full-area coverage

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent applies optical flow estimation only to edge areas where luminance changes occur, rather than performing it across the entire image. For non-edge areas, a simpler temporal integration method is used. This partial application of the computationally intensive method reduces the overall computational load while still achieving complete image restoration

Inventive Principle:

Principle #16Partial or excessive action

2Area of stationary object

If simultaneous optical flow and luminance estimation is performed to restore luminance images across the entire photographed surface, then the luminance estimation coverage is improved, but the computational load increases

Engineering Contradiction:

Improveluminance estimation coverage areaVSAvoidcomputational load

Core Design Contradiction:

Area of stationary objectVSPower

Solution Approach 1:

The patent divides the image restoration process into two distinct stages: edge detection phase (using optical flow for areas with luminance changes) and non-edge area restoration phase (using temporal integration of event data). This segmentation allows different processing methods to be applied to different regions, reducing overall computational complexity while maintaining full-area coverage

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent applies optical flow estimation only to edge areas where luminance changes occur, rather than performing it across the entire image. For non-edge areas, a simpler temporal integration method is used. This partial application of the computationally intensive method reduces the overall computational load while still achieving complete image restoration

Inventive Principle:

Principle #16Partial or excessive action

Data Source

PatentUS12315216B2Image restoration device, image restoration method, and image restoration program

Publication Date: 2025.05.27 DENSO WAVE INC

AI summary

The image restoration device has an initialization block that initializes the luminance value of each pixel coordinate to an intermediate value in the luminance array list that stores one of a pair of polarity values and intermediate values as the luminance value for each pixel coordinate. The image restoration device also has an update block that updates the initialized luminance array list according to the pixel coordinates and polarity values for each event, and an output block that outputs the luminance array list updated by the update block over the shooting period as a binary image. By the update performed in the update block, the luminance values of the firing coordinates where the event fired in the luminance array list are overwritten by the polarity values of the event. In addition, the update preserves the luminance values of the non-firing coordinates in the luminance array list, excluding the firing coordinates.