Image Recovery Behind Transparent Display via Deconvolution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The quality of images captured by a camera placed behind a transparent display is degraded due to optical disturbances such as reflection and diffraction caused by the display's wiring structure, leading to artifacts like blurring and flare, for which there are no effective solutions.
Innovation Solution
A method and device that recover images by acquiring the image data from the image sensor, obtaining point spread functions corresponding to dots on a calibration chart, and performing de-convolution using these functions to mitigate the optical disturbances.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Area of stationary object
If a camera is placed behind a transparent display to achieve a true all-display design, then the display area can be maximized and screen-to-body ratio improved, but image quality is degraded due to optical disturbances such as diffraction and reflection from the display's wiring structure
Solution Approach 1:
The patent captures a reference image through the display before capturing the actual target image. This preliminary reference image contains the optical distortion characteristics (diffraction patterns, reflection artifacts) introduced by the display's wiring structure. By having this reference data available beforehand, the system can later remove these distortions from the captured images using image processing algorithms, thus resolving the quality degradation caused by placing the camera behind the display.
Solution Approach 2:
The patent converts the harmful optical disturbances (diffraction and reflection patterns) into a useful reference. By capturing the display's distortion characteristics through a reference image, the system transforms the previously harmful artifacts into a calibration dataset that enables the removal of these same artifacts from subsequent images, thereby improving overall image quality while maintaining the all-display design.
2Manufacturing precision
If image processing is performed to remove diffraction artifacts, then image quality can be improved, but computational complexity and processing time increase
Solution Approach 1:
The patent performs the complex computational work of characterizing optical distortions in advance by capturing a reference image. This preliminary computation extracts the diffraction and reflection patterns once, storing them as a reference model. When actual images need processing, the system only needs to compare and subtract these pre-computed patterns rather than performing full deconvolution in real-time, significantly reducing the computational burden during actual image capture and processing.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
The method effectively improves image quality by eliminating artifacts caused by diffraction, allowing the camera to be placed anywhere behind the transparent display without compromising user experience, maximizing screen-to-body ratio, and enabling natural eye-point selfies without costly computations.
Implementation Method 1
The disturbance is mainly due to reflection at the display or diffraction by a wiring structure of the display
Implementation Method 2
The disturbance is mainly due to reflection at the display or diffraction by a wiring structure of the display
Implementation Method 3
recovering the image by performing a de-convolution of the image based on the plurality of point spread functions
Data Source
AI summary
A method for recovering an image passing through a display and taken by an image sensor disposed on a rear side of the display, the method includes acquiring the image taken by the image sensor, obtaining a plurality of point spread functions corresponding one-to-one to a plurality of dots arranged in a predetermined pattern on a calibration chart, and recovering the image by performing a de-convolution of the image based on the plurality of point spread functions.