Under-display camera shading compensation via adjustment matrix
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Image capture devices with under-display cameras face challenges such as display shading, light attenuation, and scattering due to the placement of camera sensors beneath or adjacent to displays, leading to reduced image quality and limited display size optimization.
Innovation Solution
The implementation of image processing techniques that involve determining an adjustment matrix based on display content, aging factors, and preset parameters to compensate for display shading, and the use of UI modes that optimize transmittance in low light conditions, allowing for improved light capture and display size maximization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Area of stationary object
If the camera sensor is placed under the display to maximize display size, then the display area is increased, but the image quality deteriorates due to display shading and light attenuation
Solution Approach 1:
An adjustment matrix is introduced as an intermediary computational layer between the degraded image data from the under-display camera and the final output image. This matrix, derived from capture data taken through the display, acts as a mediator that compensates for the shading and attenuation effects, allowing the camera sensor to remain under the display while maintaining image quality
Solution Approach 2:
The system changes the parameters of the captured image data by applying the adjustment matrix, which modifies pixel values to compensate for display-induced degradation. Additionally, the display transparency parameters are dynamically adjusted based on pixel values and aging factors to optimize light transmission to the camera sensor
2Area of stationary object
If the camera sensor is placed under the display, then display size is maximized, but light transmission to the sensor deteriorates due to attenuation and scattering
Solution Approach 1:
Capture data is collected in advance through the display under various conditions, and the adjustment matrix is pre-computed based on this data. This preliminary characterization of the display's optical properties allows for real-time compensation without requiring additional hardware modifications to the display structure
Solution Approach 2:
The system dynamically changes display parameters including transparency settings and pixel values to optimize light transmission. The adjustment matrix applies parameter transformations to the captured image data to compensate for reduced illumination intensity caused by the display layers
3Adaptability or versatility
If display content is shown over the camera sensor, then the display functionality is maintained, but image capture quality deteriorates due to scattered light
Solution Approach 1:
The adjustment matrix serves as an intermediary that separates the display functionality from its detrimental effect on image capture. By characterizing the display's scattering effects through capture data and encoding this in the adjustment matrix, the system can maintain full display functionality while computationally removing the scattered light artifacts from captured images
Solution Approach 2:
The adjustment matrix is dynamically updated based on aging factors and current display content. As the display ages or changes content, the capture data and corresponding adjustment matrix are recalibrated to maintain optimal compensation performance, allowing the system to adapt to changing conditions
4Illumination intensity
If the display transparency is increased to improve light transmission, then image capture is improved, but the display visibility and contrast deteriorate
Solution Approach 1:
The system dynamically adjusts display transparency parameters based on the operational context. During image capture, transparency is increased to improve light transmission to the sensor. During normal display operation, transparency is optimized for visibility and contrast. The adjustment matrix compensates for the transient reduction in visibility during capture
Solution Approach 2:
The adjustment matrix acts as an intermediary that decouples the relationship between display transparency and image quality. This allows the display to operate at optimal visibility settings during normal use while still achieving sufficient light transmission during capture through computational compensation
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
Enhances image quality by compensating for display shading and optimizing light transmittance, enabling larger display sizes without compromising image capture performance.
Implementation Method 1
light passes through a display layer before being received by the camera sensor
Implementation Method 2
the camera sensor may also capture light scattered by the displayed content
Implementation Method 3
layers of the display shade the camera sensor, so that less intensive and less accurate image information is received by the camera sensor
Implementation Method 4
sub-pixels above the camera sensor may cause shading, such as spatial strides and shadows
Data Source
AI summary
An example image capture device includes a display configured to display captured images, a camera sensor, the camera sensor being disposed to receive light through at least a portion of the display, memory configured to store captured images, and one or more processors coupled to the camera sensor, the display, and the memory. The one or more processors are configured to receive a signal from a sensor. The one or more processors are configured to determine, based at least in part on the signal, a user interface mode. The user interface mode includes a first mode having a first number of black pixels or a second mode having a second number of black pixels. The first number is greater than the second number. The one or more processors are also configured to receive image data from the camera sensor.