Under-Display Camera PSF Reconstruction Through Semi-Transparent Displays
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The integration of complex front-facing camera systems in electronic devices, such as depth cameras, reduces the resolution and viewing area of the display, and disposing the camera behind the display panel degrades image quality.
Innovation Solution
Generate a high dynamic range (HDR) point spread function (PSF) image and determine a spatially invariant low-resolution PSF image for image reconstruction by capturing PSFs through a semi-transparent display panel using magnifying optical elements and applying deconvolution techniques.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Area of stationary object
If the front-facing camera system is disposed behind the display panel, then the display resolution and viewing area are improved, but the image quality captured by the camera is degraded
Solution Approach 1:
The patent introduces a virtual aperture as an intermediary element that mediates between the display panel and the camera system. By controlling which pixels are transparent (aperture pixels) and which are opaque (shielding pixels), the virtual aperture enables the camera to capture images through the display while maintaining image quality. This resolves the contradiction by creating a selective transmission path that improves both display area utilization and image capture quality.
Solution Approach 2:
The patent segments the display pixels into two functional groups: aperture pixels that allow light transmission for camera capture, and shielding pixels that block light to prevent degradation. This segmentation enables the display to simultaneously serve as both a full-screen display and a selective optical window for the rearward-facing camera, resolving the contradiction between maximizing display area and maintaining image quality.
2Adaptability or versatility
If complex front-facing camera systems (e.g., depth cameras) are integrated, then camera functionality is improved, but the display resolution and viewing area are reduced
Solution Approach 1:
The patent implements a nested configuration where the camera system is positioned behind the display panel, and the display panel itself serves as the front-facing interface. The virtual aperture mechanism is nested within the display structure, allowing the camera functionality to be embedded without compromising the external display area. This nesting resolves the contradiction by hiding the complex camera system within the display structure while maintaining full display visibility.
Solution Approach 2:
The patent makes the display panel multi-functional by enabling it to simultaneously serve as the front-facing display interface and as a selective optical window for the camera system. The virtual aperture mechanism allows the same physical structure to perform both display and camera aperture functions, eliminating the need for separate physical space for the camera and resolving the area conflict.
3Measurement precision
If multiple PSF images are captured at different exposure times, then the HDR PSF accuracy is improved, but the processing time and complexity increase
Solution Approach 1:
The patent performs preliminary action by capturing multiple PSF images at different exposure times in advance, before the actual image reconstruction is needed. These pre-captured PSF images are stored for later use, allowing the system to quickly retrieve and apply the appropriate PSF without performing time-consuming measurements at the moment of reconstruction. This resolves the contradiction by shifting the time investment to a preliminary phase.
Solution Approach 2:
The patent changes the exposure time parameter when capturing PSF images, taking multiple measurements with varying exposure times to build a comprehensive HDR PSF model. By systematically varying this parameter and storing the results, the system achieves high measurement precision while enabling fast retrieval during actual operation, resolving the contradiction between accuracy and processing time.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
Enhances display resolution and viewing area, allowing full-screen graphical user interfaces and improved image quality by reconstructing blurred images captured behind the display panel.
Implementation Method 1
measuring a high resolution point spread function image of the electronic device through one or more magnifying optical elements disposed between the camera lens and the image sensor
Data Source
Figure 1a
Figure 1b
Figure 2
AI summary
A method for controlling an electronic device is disclosed. The method comprise: obtaining, by a camera, a plurality of point spread function (PSF) images by capturing a semi-transparent pixel region of a display panel of the electronic device for a plurality of exposure times; determining, for each of the plurality of PSF images, pixel intensity data for each of a plurality of pixel locations of the PSF images; calculating, for the plurality of the pixel locations, a weighted average pixel intensity value based on the pixel intensity data and the exposure time corresponding to the plurality of the pixel locations; and generating a high dynamic range (HDR) PSF image using the weighted average pixel intensity value.