Under-Display Camera Image Interference Mitigation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The placement of digital cameras underneath device displays in modern devices, such as smartphones, interferes with image capture due to light emission from the display, causing interference and corruption of captured images.
Innovation Solution
The use of calibration images to determine capture parameters and generate subtraction images, which are then used to mitigate display interference by adjusting for the light emitted by the display, along with recalibration images to adapt to changes in display alignment and recalibration, and controlling display brightness to minimize interference.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Area of stationary object
If the digital camera is positioned underneath the display to maximize screen area, then the display area is increased, but light from the display interferes with image capture quality
Solution Approach 1:
The system performs preliminary calibration by capturing calibration images with the display showing specific patterns (e.g., uniform colors, gradients) before actual image capture. These calibration images are used to pre-compute a mapping or lookup table that characterizes the display's light emission characteristics, which is then applied during actual photography to correct the interference.
Solution Approach 2:
The patent converts the harmful display light emission into a useful calibration signal. By intentionally capturing images with known display patterns, the system characterizes the interference and uses this knowledge to subtract or correct the display's contribution from subsequent photos, effectively turning the interference source into a reference for correction.
2Measurement precision
If calibration images are captured for each pixel type (red, green, blue) to accurately mitigate interference, then image accuracy is improved, but processing complexity increases
Solution Approach 1:
The calibration process is segmented into separate captures for each primary color (red, green, blue) displayed on the screen. This allows the system to independently characterize the emission characteristics of each color channel, enabling more accurate correction of the composite image by treating each color component separately through the use of color filter arrays or spectral decomposition.
Solution Approach 2:
The system varies display parameters (color patterns, brightness levels, refresh rates) during calibration to characterize the display's emission under different conditions. This includes capturing calibration images at multiple brightness levels and color combinations to build a comprehensive model of the display's light output, which is then used to adaptively correct images under varying operating conditions.
3Manufacturing precision
If the display brightness is controlled to minimize interference, then image capture quality is improved, but display visibility and user experience are reduced
Solution Approach 1:
The system employs periodic or pulsed display activation during calibration, where the display rapidly cycles between different brightness levels or color states. This allows the camera to capture calibration data at controlled intervals while maintaining overall display visibility. During actual image capture, the display may be temporarily dimmed or strobed in synchronization with the shutter speed to minimize interference while preserving user experience.
Solution Approach 2:
The display brightness is dynamically adjusted based on the capture context. During calibration, the display shows specific patterns at controlled brightness levels. During actual photography, the system dynamically modulates the display brightness or content in real-time coordination with the camera exposure, allowing the display to remain visible to the user while minimizing light interference during the brief capture moment.
Data Source
AI summary
A device includes a digital camera positioned underneath a display of the device that captures images through the display. When capturing images using the digital camera, light emitted by the display can get into the digital camera, interfering with or corrupting the captured images. An enhanced output image is generated from a captured image that mitigates the interference from the display. The enhanced output image is generated, for example, by capturing a different calibration image for each pixel type (e.g., Red, Green, and Blue) included in the display. When an image is subsequently captured, an amount of color emitted by the pixels on the display for the currently displayed image is determined and one or more subtraction images are generated and subtracted from the captured image to mitigate the interference from the display.