Light Field Image Reconstruction for Scene Browsing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional image capturing techniques, such as Light Field Photography, generate image data that cannot be browsed to recognize objects before setting a focal length, making it difficult for users to specify scenes with desired objects.
Innovation Solution
An image processing apparatus that obtains image signals from an image capturing device, sets a focal length for objects to be focused, generates reconstructed images by adding pixel values corresponding to specific pixels, and outputs a moving image by concatenating images at various focal lengths, allowing users to specify objects in a scene.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If thumbnail images are generated focused at one focal length from LF data, then the user can choose LF data which captured a desired scene, but images of objects focused at focal lengths different from that focal length are blurred, making the user unable to recognize the objects
Solution Approach 1:
The patent segments the light field data by generating multiple thumbnail images, each focused at a different focal length. This segmentation allows users to browse through different focal planes to locate scenes of interest, while the underlying LF data retains all focal information for later reconstruction of sharp images at any desired focal length.
Solution Approach 2:
The patent introduces an intermediary representation (thumbnail images at various focal lengths) that mediates between the raw LF data and the user's need to recognize objects. These thumbnails serve as a browsing interface that preserves object recognizability at each focal plane while not losing the underlying LF information.
2Reliability
If LF data is recorded without generating focused images, then all light flux information is preserved, but the user cannot recognize the contents of the object before setting a focal length
Solution Approach 1:
The patent performs preliminary action by generating thumbnail images at multiple focal lengths during or after the imaging process. These pre-generated thumbnails provide immediate visual feedback about scene contents at different focal planes, enabling users to browse and select scenes of interest without losing the complete LF data for later high-quality reconstruction.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
Enables users to recognize and specify objects in a scene by generating clear, focused images at arbitrary focal lengths, improving the usability of image data recorded by image capturing devices.
Implementation Method 1
focusing light fluxes which have passed through different pupil regions of an imaging lens on respective pixels (photoelectric converters) of an imaging element via a microlens array
Implementation Method 2
neighboring pixels record light fluxes entering from different directions, and light fluxes in the same direction are extracted from pixels associated with respective microlenses
Implementation Method 3
outputs from pixels which record light fluxes that have passed through one point on a focal plane of that focal length are added, thereby generating (reconstructing) pixels of an image focused at the specific focal length after imaging
Data Source
AI summary
An image processing apparatus obtains an image signal, which is captured by an image capturing apparatus, and in which respective pixels correspond to light fluxes of different combinations of pupil regions, where the light fluxes have passed through in an imaging optical system of the image capturing apparatus, and incident directions. The image processing apparatus sets a focal length corresponding to an object to be focused, and generates a reconstructed image focused on the object of the set focal length from the image signal. The image processing apparatus generates a moving image by concatenating a plurality of reconstructed images generated in association with a plurality of different focal lengths, and outputs the moving image in association with the image signals.