TOF Camera Dead Pixel Restoration via Light Intensity Map

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing time-of-flight (TOF) cameras face reliability issues in generating depth images, particularly at image boundaries affected by lighting or shadows, leading to unreliable depth values and dead pixels.

Innovation Solution

A method and apparatus that restore dead pixels in depth maps using both depth and light intensity information acquired through a TOF camera, by forming a similarity-based region in the light intensity map to compensate for dead pixels, thereby enhancing image clarity and minimizing lighting or shadow effects.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Device complexity

If depth image is generated only by using depth map, then the generation process is simple, but the reliability is reduced at boundary portions affected by lighting or shadows

Engineering Contradiction:

Improveimage generation processVSAvoiddepth image reliability

Core Design Contradiction:

Device complexityVSReliability

Solution Approach 1:

The patent combines depth map and light intensity map to generate the depth image. By merging these two different types of information, the system compensates for the weaknesses of using only depth map (which is unreliable at boundaries affected by lighting or shadows) while maintaining a relatively simple generation process.

Inventive Principle:

Principle #5Merging (Combining)

2Ease of operation

If depth map is used alone for depth image generation, then the processing is straightforward, but dead pixels occur at boundary portions

Engineering Contradiction:

Improveimage generation operationVSAvoiddepth value accuracy

Core Design Contradiction:

Ease of operationVSManufacturing precision

Solution Approach 1:

The light intensity map serves as an intermediary to restore dead pixels in the depth map. By using the light intensity information as a mediator, the system can identify and restore dead pixels at boundary portions without significantly complicating the overall operation.

Inventive Principle:

Principle #24Intermediary (Mediator)

3Device complexity

If only depth map is utilized, then the data processing is simple, but lighting and shadow effects reduce image quality

Engineering Contradiction:

Improvedata processing complexityVSAvoidlighting and shadow effects

Core Design Contradiction:

Device complexityVSObject-affected harmful factors

Solution Approach 1:

The patent converts the harmful effect of lighting and shadows into a beneficial factor by using the light intensity map. The same lighting conditions that cause dead pixels in the depth map are utilized to identify and restore those dead pixels, thereby converting a harmful factor into a useful tool for improving image quality.

Inventive Principle:

Principle #22Blessing in disguise (Convert harm into benefit)

Applied Scientific Principles

This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.

Function Achieved in This Case

The solution effectively restores dead pixels, ensuring clear display of image boundaries and improving the reliability of depth images by integrating depth and light intensity information, thus addressing the limitations of TOF cameras in handling lighting and shadow impacts.

Implementation Method 1

a depth image has been generated only by using a depth map for the image... acquired through a TOF-type camera... using time-of-flight

Methodology Applied

Scientific EffectTime of flight: Time of Flight

Implementation Method 2

each pixel value in a two-dimensional pixel region constituting the light intensity information is expressed by a light intensity value reflected from the subject with respect to the light illuminated from the emitter

Methodology Applied

Scientific EffectLight reflection: Reflection

Data Source

PatentUS8654219B2Method and apparatus for restoring dead pixel using light intensity map in a time-of-flight camera

Publication Date: 2014.02.18 LG ELECTRONICS INC

AI summary

Disclosed herein is an image display apparatus including a display unit; a camera unit configured to capture a subject, and to acquire a depth image from the captured subject; and a controller operatively connected to the display unit and the camera unit, the controller configured to control the camera unit to restore an interest region based on depth information and light intensity information when the interest region exists in the acquired depth image.