Camera Behind Display Image Reconstruction Using Multiple PSFs

Solution Overview

Problem

Electronic devices with front-facing cameras disposed behind the display panel experience image degradation due to the camera's position, leading to reduced display resolution and viewing area, necessitating improved image reconstruction techniques.

Innovation Solution

The electronic device captures images through a semi-transparent display panel and uses multiple premeasured point spread functions (PSFs) at different depth and lateral positions to reconstruct images by deconvolving and stitching image patches for each color component, enhancing image quality and resolution.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Area of stationary object

If the front-facing camera system is disposed behind the display panel, then the display resolution and viewing area are improved, but the image quality captured by the camera deteriorates

Engineering Contradiction:

Improvedisplay viewing areaVSAvoidimage quality

Core Design Contradiction:

Area of stationary objectVSManufacturing precision

Solution Approach 1:

The display panel is segmented into transparent regions and opaque regions, allowing the camera to be positioned behind the transparent region while maintaining display functionality in other areas. This segmentation enables the camera system to be integrated without compromising overall display area or image quality in visible regions.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

A light guide layer is introduced as an intermediary component between the camera and display panel. This light guide layer redirects light from the camera through the display panel, compensating for the image degradation caused by the camera's position behind the display and restoring image quality.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Measurement precision

If the front-facing camera system is disposed behind the display panel, then the display resolution is improved, but the image quality captured by the camera deteriorates

Engineering Contradiction:

Improvedisplay resolutionVSAvoidimage quality

Core Design Contradiction:

Measurement precisionVSManufacturing precision

Solution Approach 1:

The display panel is segmented into transparent regions and opaque regions, allowing the camera to be positioned behind the transparent region while maintaining display functionality in other areas. This segmentation enables the camera system to be integrated without compromising overall display area or image quality in visible regions.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

A light guide layer is introduced as an intermediary component between the camera and display panel. This light guide layer redirects light from the camera through the display panel, compensating for the image degradation caused by the camera's position behind the display and restoring image quality.

Inventive Principle:

Principle #24Intermediary (Mediator)

3Adaptability or versatility

If the camera system area is expanded to increase complexity (e.g., depth cameras), then the camera functionality is improved, but the display area is reduced

Engineering Contradiction:

Improvecamera functionalityVSAvoiddisplay area

Core Design Contradiction:

Adaptability or versatilityVSArea of stationary object

Solution Approach 1:

The camera system is nested within the display structure by positioning it behind the display panel. This nesting allows the camera system to be integrated into the existing display architecture without requiring additional external space, thereby maintaining full display area while enabling enhanced camera functionality including depth sensing.

Inventive Principle:

Principle #7Nested doll (Nesting)

Solution Approach 2:

The camera system is moved from a lateral positioning (adjacent to the display) to a depth positioning (behind the display panel). This dimensional change allows the camera system to be integrated without consuming lateral display area, enabling enhanced camera functionality while preserving full display viewing area.

Inventive Principle:

Principle #17Another dimension (Dimensionality change)

Applied Scientific Principles

This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.

Function Achieved in This Case

This approach increases the display's viewing area and resolution, allowing for full-screen views and improved graphical user interfaces, enabling higher resolution images and flexible camera placement, such as in the center of the display, enhancing user experience and aesthetic quality.

Implementation Method 1

capturing an original image through a semi-transparent pixel region of the display panel

Methodology Applied

Scientific EffectLight transmission through semi-transparent material: Absorption (EM radiation)

Implementation Method 2

reconstruct images by deconvolving and stitching image patches for each color component

Methodology Applied

Scientific EffectDeconvolution:

Data Source

PatentUS11721001B2Multiple point spread function based image reconstruction for a camera behind a display

Publication Date: 2023.08.08 SAMSUNG ELECTRONICS CO LTD

AI summary

A method includes capturing, by a camera disposed behind a display panel of an electronic device, an original image through a semi-transparent pixel region of the display panel, and determining a depth position with respect to at least one object identified within the original image. The method further includes accessing, based on the depth position, a plurality of point spread functions (PSFs) corresponding to a plurality of lateral positions at the depth position, and generating a set of image patches based on the plurality of PSFs. Each image patch of the set of image patches is generated based on a different one of the plurality of PSFs. The method concludes with generating a reconstructed image corresponding to the original image based on the set of image patches.