Under-Display Camera Artifact Correction Using Brightness Mapping
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing electronic apparatuses with cameras under display (CUD) methods suffer from artifacts due to reflection and diffraction of light through display pixels, which cannot be effectively removed by general image processing, and estimation accuracy deteriorates due to varying imaging conditions between captured and target images.
Innovation Solution
An electronic apparatus with a camera installed on the display's back surface uses a machine learning model to determine high-brightness areas, estimate artifact distributions, and apply correction images based on learned relationships between images with and without display transmission, combining these to improve image quality.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Area of stationary object
If the camera is installed on the back surface of the display (CUD method), then the screen occupancy rate is improved, but artifacts occur due to light reflection and diffraction through display pixels
Solution Approach 1:
The patent converts the harmful artifacts caused by light reflection and diffraction into a predictable pattern that can be removed through image processing. By capturing a reference image without the display in place and using it to generate a correction map, the system transforms the harmful optical effects into correctable data, effectively eliminating the artifacts while preserving the CUD configuration benefits
Solution Approach 2:
The patent introduces a correction map as an intermediary element between the captured image and the final output. This correction map, generated from a reference image captured without the display, acts as a mediator that identifies and removes artifact patterns caused by light interaction with display pixels, allowing the CUD camera to function effectively despite the inherent optical interference
2Device complexity
If general image processing methods are used to remove artifacts, then processing simplicity is maintained, but artifact removal effectiveness is insufficient
Solution Approach 1:
The patent performs preliminary action by capturing a reference image before actual imaging operations. This reference image, taken without the display in place, pre-establishes the artifact pattern that will be present during normal operation. By preparing this correction data in advance, the system enables effective artifact removal during actual imaging without requiring complex real-time processing algorithms
3Measurement precision
If a machine learning model is used to estimate correction images, then artifact removal accuracy is improved, but estimation accuracy deteriorates when imaging conditions differ between training and actual use
Solution Approach 1:
The patent extracts the artifact pattern from a reference image captured under controlled conditions (without the display) and separates it from the actual imaging data. By isolating the artifact component through subtraction operations, the system creates a correction map that can be applied to remove artifacts from subsequent images, even when lighting and other conditions vary, thus improving reliability across different imaging scenarios
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
The solution effectively removes artifacts while maintaining image quality by determining and correcting brightness and artifact distributions, enhancing the accuracy of estimated images.
Implementation Method 1
light transmitted through a part of an area of the display is incident on the camera
Implementation Method 2
pixels disposed in the imaging area are turned off. A corresponding portion of incident light from an object is reflected on a surface of the pixel disposed on a display panel
Implementation Method 3
The reflection and diffraction can be a main cause of an artifact that is likely to occur around an image of a light source
Data Source
AI summary
An electronic apparatus includes: a display in which a plurality of pixels are arranged; a camera; and a controller. Light transmitted through a part of an area of the display is incident on the camera. The controller is configured to determine a high-brightness area in which a brightness exceeds a certain value, based on a brightness distribution of a captured image captured by the camera, determine an artifact distribution based on an artifact pattern for a point light source and the high-brightness area, estimate a correction image based on the captured image by using a machine learning model that has learned a relationship between a first image captured with transmission through the display and a second image captured without transmission through the display, determine an applied intensity of the correction image based on the artifact distribution, and combine the correction image and the captured image based on the applied intensity.