Interactive Refocusing via Duration Focal Volume and Space-Time Index Map
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional imaging techniques face limitations in controlling the depth of field (DOF), leading to a fixed and single DOF that cannot be altered after capturing an image, resulting in objects within or outside the DOF being in-focus or blurred, respectively, and failing to capture multiple depths of field simultaneously.
Innovation Solution
A system and method that captures a series of images over time with varying DOFs, concatenates them to create a duration focal volume, computes a space-time in-focus image, and generates a space-time index map to identify in-focus areas, allowing for interactive refocusing and extended DOF.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If the aperture is decreased to increase depth of field, then more of the scene appears in focus, but less light is received at the image sensor requiring longer exposures which decrease signal to noise ratio and cause blurring
Solution Approach 1:
The system performs preliminary actions by capturing multiple images at different focal depths before the user views the final result. During playback, the system identifies in-focus pixels from the appropriate focal depth for each pixel location, effectively performing the refocusing action in advance and allowing instantaneous interaction without requiring long exposures at small apertures
Solution Approach 2:
The invention adds a temporal dimension to the imaging process by capturing images across multiple focal depths over time, creating a four-dimensional dataset (x, y, focal depth, time). This allows the system to select the optimal focal depth for each pixel during playback, resolving the contradiction between depth of field and signal to noise ratio by accessing data from different focal planes
2Measurement precision
If pixel size decreases or sensor resolution increases, then image resolution improves, but depth of field decreases and signal to noise ratio falls
Solution Approach 1:
The system captures multiple images at different focal depths in advance, storing the focal depth information for each pixel. During playback, it retrieves the pre-computed in-focus image data, allowing high-resolution displays to show images at the appropriate focal depth without suffering from reduced depth of field
Solution Approach 2:
The invention changes the parameter of focal depth dynamically for each pixel based on the display resolution and user interaction. By capturing data at multiple focal depths and selecting the optimal one during playback, the system adapts to different display resolutions while maintaining both image quality and appropriate depth of field
3Ease of operation
If a single depth of field is captured, then the image sensor can only focus on one range in the scene, but objects outside the depth of field are blurred and cannot be refocused
Solution Approach 1:
The system performs preliminary capture of multiple focal depth images during the shooting phase. This preliminary action creates a library of in-focus data at different depths, enabling versatile refocusing operations during playback without requiring complex post-processing or additional shooting
Solution Approach 2:
The invention creates copies of the scene at different focal depths by capturing multiple images. During playback, the system selects and displays the appropriate copy (in-focus image) for each pixel location, enabling refocusing to different depths while maintaining the simplicity of single-shot capture
Data Source
AI summary
Systems, methods, and media for providing interactive refocusing are provided, the systems comprising: a lens; an image sensor; and a processor that: causes the image sensor to capture a plurality of images over a predetermined period of time, wherein each of the plurality of images represents a scene at a different point in time; changes a depth of field between at least a pair of the plurality of images; concatenates the plurality of images to create a duration focal volume in the order in which the images were captured; computes a space-time in-focus image that represents in-focus portions from each of the plurality of images based on the duration focal volume; and computes a space-time index map that identifies an in-focus image for each location of the scene from among the plurality of images based on the duration focal volume and the space-time in-focus image.