Under-Display Camera Training Data Capture With Moving Display
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The integration of front-facing cameras with electronic displays in devices results in unsightly borders and limited screen-to-body ratio, and image capture technologies face interference and noise due to the integration, necessitating improved image processing techniques to enhance usable display area.
Innovation Solution
A system that moves an electronic display between positions to cover or uncover the camera's field-of-view, using an automaton to capture training data for machine learning models to correct image distortions caused by the display.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the front-facing camera is disposed within a notch or cutout of the display, then the camera has an unobstructed view of the environment, but the electronic display has large unusable portions and reduced screen-to-body ratio
Solution Approach 1:
The patent extracts the camera from the traditional front position and relocates it to the back of the device, allowing the front display to be continuous and unobstructed. The back cover is designed with a camera assembly that provides the necessary viewing capability without compromising the front display area.
Solution Approach 2:
The camera is moved from the front plane to the back plane of the device, utilizing the third dimension (depth) to resolve the conflict between camera positioning and display continuity. This spatial reorganization allows both functions to coexist without interference.
2Device complexity
If the front-facing camera is integrated on the same surface as the electronic display, then the device achieves greater integration, but image capture quality deteriorates due to noise and interference
Solution Approach 1:
The camera is extracted from the display surface and positioned on the back cover, eliminating the physical and electromagnetic interference between the camera sensor and display pixels. This separation resolves the image quality degradation caused by integration.
Solution Approach 2:
The patent introduces software processing and image algorithms as intermediaries to compensate for any remaining interference effects. The system uses computational photography techniques to enhance image quality and reduce noise from the integrated design.
3Area of stationary object
If the electronic display covers the camera's field-of-view during operation, then the usable display area is maximized, but training data for image restoration models becomes inconsistent
Solution Approach 1:
The patent employs a dynamic training data collection system where the display is moved between covering and uncovered positions relative to the camera. This dynamic approach allows the collection of diverse training images under varying conditions, improving the robustness and consistency of the restoration model.
Solution Approach 2:
The system performs preliminary actions by collecting training data under controlled conditions before actual operation. Images are captured in advance with the display in known positions, allowing the restoration model to learn the specific distortion patterns and compensate for them during normal use.
Data Source
AI summary
This disclosure provides methods, devices, and systems for machine learning. The present implementations more specifically relate to automatons that can acquire input images and ground truth images for training neural network models. In some aspects, a system for acquiring training data may include a camera, an electronic display, and an apparatus configured to maintain the camera in a stationary position while moving the electronic display in and out of the camera's field-of-view (FOV). In some aspects, the system may further include a controller configured to acquire training data via the camera based on the positioning of the electronic display. In some implementations, the controller may acquire ground truth images of a scene while the electronic display is covering the camera's FOV. In some other implementations, the controller may acquire input images of the scene while the electronic display is outside the camera's FOV.