Image Sensor Data Processing With Selective Blur for Real-Time Bokeh
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Modern portable electronic devices, such as smartphones, are unable to optically generate bokeh effects due to their small lenses and limited apertures, and existing software-based simulations are slow and computationally intensive, making them unsuitable for real-time applications like viewfinder modes.
Innovation Solution
A method involving convolution of a blur kernel with image sensor data to create simulated bokeh effects, utilizing a predefined shape and increasing pixel intensity in saturated regions, combined with downscaling and upscaling techniques to accelerate the process, allowing for realistic bokeh simulation in both still images and viewfinder modes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If software-based bokeh simulation is implemented in modern portable electronic devices, then bokeh effect can be achieved, but the processing speed is slow and computational overhead is high
Solution Approach 1:
The image processing is divided into multiple passes: first identifying saturated pixels (light sources), then applying selective blur only to those regions using convolution with a blur kernel, while leaving other regions sharp. This segmentation approach reduces overall computational load compared to blurring the entire image.
Solution Approach 2:
Different processing is applied to different regions of the image: saturated pixels receive blur treatment to create bokeh effect, while non-saturated regions remain sharp and in focus. This local differentiation maintains image quality in important regions while applying the aesthetic effect only where needed.
2Reliability
If traditional computational bokeh methods are used, then bokeh effect is produced, but the process is too slow for real-time applications like viewfinder mode
Solution Approach 1:
The algorithm first identifies and marks saturated pixels before applying the blur effect. By pre-processing to locate light sources and prepare a mask of regions requiring blur, the subsequent convolution operation is optimized to process only relevant areas, reducing overall processing time for real-time display.
Solution Approach 2:
Instead of applying full-image blur or complex optical simulations, the method applies blur selectively only to saturated pixel regions. This partial action approach achieves the essential bokeh effect (blurred light sources) without the computational cost of processing the entire image at high resolution.
Data Source
AI summary
An electronic device includes an image capture device configured to capture image sensor data in response to initiation of an image capture operation. The electronic device includes one or more processors and a display operable with the one or more processors. The one or more processors select saturated pixels of the image sensor data and determine a gain factor as a function of brightness values of pixels in the image sensor data. The one or more processors increase an intensity value of the saturated pixels by the gain factor to create scaled image sensor data, blur at least some of the scaled image sensor data by convolving a blur kernel with the scaled image sensor data to create altered image sensor data exhibiting a simulated bokeh effect and cause the display to present the altered image sensor data exhibiting the simulated bokeh effect.