Mobile Device Bokeh Effect via Computational Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Mobile device cameras struggle to capture high-quality portraits with a blurred background, known as the bokeh effect, due to intrinsic limitations in lens quality and size, which is typically achieved by professional DSLR cameras.
Innovation Solution
The implementation of machine learning techniques and GPU computation power on mobile devices to automatically and real-time enhance images by applying a bokeh effect, utilizing object detection, segmentation, and parallel processing to mimic DSLR camera performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If a high-end lens with a wide-open aperture is used to produce the bokeh effect, then the quality of the blurred background is improved, but the device complexity and cost increase significantly
Solution Approach 1:
The patent replaces the mechanical optical system (high-end lens with wide-open aperture) with a computational system using machine learning models and GPU processing. The bokeh effect is achieved through software-based image processing that analyzes and blurs background regions algorithmically, eliminating the need for complex optical hardware while maintaining visual quality
Solution Approach 2:
The patent creates a computational copy of the bokeh effect that mimics the visual output of expensive DSLR cameras. By training machine learning models on examples of optical bokeh, the system replicates the characteristic blurred background appearance without requiring the physical lens characteristics that produce it
2Speed
If real-time processing is implemented to enhance user experience, then the responsiveness is improved, but the computational power and energy consumption increase
Solution Approach 1:
The patent segments the image processing task into distinct stages: object detection and segmentation to identify foreground subjects, followed by selective background processing. This division allows the system to focus computational resources only on regions requiring bokeh processing, reducing overall energy consumption while maintaining real-time performance
Solution Approach 2:
The patent introduces an intermediary object segmentation mask that separates foreground objects from background regions. This mask acts as a mediator that guides the bokeh processing, allowing the system to apply computationally intensive blur operations only to background pixels while leaving foreground pixels unchanged, thereby reducing total energy consumption
Data Source
AI summary
Systems and methods are provided for receiving an image from a camera of a mobile device, analyzing the image to determine a subject of the image, segmenting the subject of the image to generate a mask indicating an area of the image comprising the subject of the image, applying a bokeh effect to a background region of the image to generate a processed background region, generating an output image comprising the subject of the image and the processed background region, and causing the generated output image to display on a display of the mobile device.