Light Field Image Processing for Edge Detection and Object Removal

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Current light field imaging techniques lack advanced methods for robust edge detection, precise object removal, and efficient transfer of depth information, limiting functionalities such as refocusing and perspective changes.

Innovation Solution

A method for processing light field data that involves rendering images at specific focal planes, applying masks to remove objects or edges, and utilizing angular information to replace obstructed pixels with background data, enabling precise edge detection and object manipulation, including copy/paste functionality that maintains depth information.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If light field data is processed using traditional 2D camera methods, then the processing is simple, but edge detection accuracy and object removal precision are insufficient

Engineering Contradiction:

Improveedge detection accuracyVSAvoidprocessing complexity

Core Design Contradiction:

Measurement precisionVSDevice complexity

Solution Approach 1:

The light field data is segmented into multiple subset images, each corresponding to a different viewpoint or direction. This segmentation allows for more precise edge detection by analyzing angular information across multiple subsets, while the modular nature of processing individual subsets keeps the overall system manageable

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent transitions from 2D image processing to 4D light field processing by incorporating angular information as an additional dimension. This enables precise edge detection and object removal by utilizing depth and angular data, while the structured approach to handling the extra dimensions prevents excessive complexity

Inventive Principle:

Principle #17Another dimension (Dimensionality change)

2Manufacturing precision

If masks are applied to remove objects from light field images, then object removal precision is improved, but information loss occurs in the processed data

Engineering Contradiction:

Improveobject removal precisionVSAvoiddata information loss

Core Design Contradiction:

Manufacturing precisionVSLoss of information

Solution Approach 1:

The patent extracts and removes only the specific angular information corresponding to the masked objects from the light field data, while preserving all other information. This selective extraction achieves precise object removal without unnecessarily discarding valuable data from other angular perspectives

Inventive Principle:

Principle #2Taking out (Extraction)

Solution Approach 2:

The patent modifies the light field data by changing the angular parameters associated with masked regions while maintaining other parameters intact. This allows precise object removal by altering only the necessary data parameters, minimizing overall information loss

Inventive Principle:

Principle #35Parameter changes

3Reliability

If multiple subset images are processed to capture angular information, then light field data completeness is improved, but processing time and computational load increase

Engineering Contradiction:

Improvelight field data completenessVSAvoidprocessing time

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The patent performs preliminary organization and preprocessing of subset images before full light field processing. By pre-arranging the subset images and identifying key angular information in advance, the system ensures complete light field data while reducing the computational burden during final processing

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The patent maintains continuous processing of light field data through efficient algorithms that handle subset images in a streamlined manner. This continuous processing approach ensures all angular information is captured while minimizing idle time and optimizing computational resource utilization

Inventive Principle:

Principle #20Continuity of useful action

4Adaptability or versatility

If copy/paste functionality is implemented for light field images, then object transfer capability is improved, but maintaining depth information and image integrity becomes more complex

Engineering Contradiction:

Improveobject transfer capabilityVSAvoidmanipulation complexity

Core Design Contradiction:

Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The patent implements copy/paste functionality by creating copies of light field data representing selected objects. These copies maintain all original light field properties including angular information and depth data, enabling versatile object transfer while the systematic copying process keeps manipulation complexity manageable

Inventive Principle:

Principle #26Copying

Data Source

PatentUS9170723B2Method, graphical user interface, and computer program product for processing of a light field image

Publication Date: 2015.10.27 SONY GROUP CORP

AI summary

A method and a graphical user interface for processing of an object in light field data is provided. The method and graphical user interface may be used to remove an object or obstruction blocking another object or background of a captured scene. The method and graphical user interface may also be used for high precision edge detection in a light field image.