Computational Depth Data Generation for Mobile Camera Refocusing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Mobile devices with integrated cameras lack the ability to capture images with a shallow depth of field, as they typically only produce all-in-focus images due to hardware size limitations, and existing refocusing techniques require special hardware that is difficult to incorporate.
Innovation Solution
A computer-implemented method for generating depth data using a camera-enabled mobile device, which captures a reference image and a sequence of images from different poses, allowing users to select and refocus portions of the image using a virtual lens model based on depth data, without requiring additional hardware.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Volume of moving object
If image capture devices are incorporated into mobile devices with small form factor, then device portability is improved, but the ability to capture images with shallow depth of field deteriorates
Solution Approach 1:
The patent replaces the mechanical/optical focus control system with a computational approach. Instead of using physical lens mechanisms to achieve shallow depth of field, the system captures multiple images from different positions and uses computer vision algorithms to synthesize refocused images with selective depth of field. This substitution allows mobile devices to achieve professional camera effects without requiring complex mechanical focus control mechanisms.
Solution Approach 2:
The patent moves from a single 2D image capture to capturing images across multiple spatial positions (adding a temporal and spatial dimension). By capturing a sequence of images as the camera moves through space and then processing them computationally, the system can selectively focus on different depth planes, effectively adding a depth dimension control that wasn't available in traditional single-shot mobile photography.
2Adaptability or versatility
If special hardware components are added to enable refocusing capability, then image refocusing ability is improved, but device complexity and difficulty of incorporation deteriorates
Solution Approach 1:
The patent replaces specialized refocusing hardware with a software-based computational imaging system. The refocusing capability is achieved through algorithms that process sequences of standard images captured by the mobile device's existing camera, eliminating the need for additional optical components, liquid lenses, or other specialized hardware while maintaining adaptability for refocusing at different depths.
Solution Approach 2:
The patent makes the mobile device's existing camera system multi-functional by enabling it to capture both standard photographs and depth-aware refocusable imagery using the same hardware. The computational processing pipeline can handle various imaging modes (standard photos, video, refocusable images) without requiring dedicated hardware for each function, thus reducing overall device complexity while maintaining versatility.
3Ease of manufacture
If refocus techniques are implemented with existing mobile device hardware, then ease of incorporation is improved, but spatial resolution of processed images deteriorates
Solution Approach 1:
The patent performs preliminary actions by capturing a sequence of high-resolution images at multiple positions before the actual refocusing operation. This pre-capture of multiple perspectives provides redundant information that can be computationally combined to maintain or even enhance spatial resolution in the final refocused image, overcoming the resolution limitations that would result from simple post-processing of a single image.
Solution Approach 2:
The patent merges multiple images captured from different positions and angles into a single refocused output image. By combining the information from multiple captures through computational processing, the system can reconstruct high-resolution details at the desired focal plane, effectively merging the strengths of each individual capture to overcome the resolution limitations of any single image.
Data Source
AI summary
Systems and methods for the generation of depth data for a scene using images captured by a camera-enabled mobile device are provided. According to a particular implementation of the present disclosure, a reference image can be captured of a scene with an image capture device, such as an image capture device integrated with a camera-enabled mobile device. A short video or sequence of images can then be captured from multiple different poses relative to the reference scene. The captured image and video can then be processed using computer vision techniques to produce an image with associated depth data, such as an RGBZ image.