Electronic Display In-Frame Sensing and Adaptive Control

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Electronic displays face challenges in sensing environmental and display-related operational parameters while maintaining image quality, particularly in dark environments where light emission during sensing can affect perceived image quality.

Innovation Solution

The electronic display performs sensing operations by illuminating sense pixels during the refresh of display pixels, allowing for simultaneous sensing and image frame display, and can continue propagating refresh pixel groups during sensing to minimize perceivable luminance variations.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If sensing operations are performed by illuminating display pixels during image frame display, then environmental and display parameters can be sensed, but light emission during sensing affects perceived image quality especially in dark environments

Engineering Contradiction:
Improvesensing accuracyVSAvoidlight emission affecting image quality
Core Design Contradiction:
Measurement precisionVSObject-affected harmful factors

Solution Approach 1:

The display panel is divided into sense pixels and non-sense pixels. Sense pixels are specifically designated for sensing operations and can be illuminated without affecting the display quality perceived by users, as they are either located in non-visible areas or their illumination is imperceptible. This segmentation allows sensing to occur without compromising image quality.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The sensing function is extracted from the display pixels and assigned to dedicated sense pixels. This separation allows the sensing operation to be performed independently from the display function, enabling parameter sensing without requiring the display pixels to emit light that would affect image quality.

Inventive Principle:
Principle #2Taking out (Extraction)

2Measurement precision

If refresh pixel groups are paused during sensing operations, then sensing can be performed accurately, but variation in light emission duration causes perceived luminance variations between different display pixels

Engineering Contradiction:
Improvesensing accuracyVSAvoidluminance uniformity
Core Design Contradiction:
Measurement precisionVSStability of the object's composition

Solution Approach 1:

The system dynamically adjusts the propagation of refresh pixel groups based on sensing requirements. During sensing operations, the refresh pixel group propagation is coordinated with the sensing timing, allowing pixels to be refreshed at different times based on whether they are sense pixels or display pixels. This dynamic adjustment enables both accurate sensing and maintenance of luminance uniformity.

Inventive Principle:
Principle #15Dynamics

Solution Approach 2:

The display refresh operation uses periodic refresh cycles, and sensing operations are integrated into specific periods of these cycles. By scheduling sensing during specific refresh periods and coordinating the illumination of sense pixels with the refresh timing, the system maintains periodic operation that ensures both sensing accuracy and luminance uniformity across all pixels.

Inventive Principle:
Principle #19Periodic action

3Productivity

If multiple refresh pixel groups are propagated simultaneously to increase refresh rate, then display performance improves, but timing complexity increases and sensing operations become more complex

Engineering Contradiction:
Improverefresh rateVSAvoidtiming complexity
Core Design Contradiction:
ProductivityVSDevice complexity

Solution Approach 1:

Multiple refresh pixel groups are segmented and assigned to different sensing and display regions. Each refresh pixel group can be independently controlled and timed, allowing the system to maintain high refresh rates while simplifying the timing coordination for sensing operations by treating each group separately rather than managing all pixels uniformly.

Inventive Principle:
Principle #1Segmentation

Applied Scientific Principles

This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.

Function Achieved in This Case

This approach enables accurate sensing of operational parameters without compromising image quality, reducing the perceivability of luminance variations and maintaining efficient display performance.

Implementation Method 1

the display pixel may be non-light emitting when refreshing. To refresh each of the display pixels, an electronic display may propagate a refresh pixel group through (e.g., down) the display pixels

Methodology Applied
Scientific EffectLight emission: Light Emitting Diode

Implementation Method 2

sensing operations may be performed by illuminating one or more display pixels (e.g., sense pixels)

Methodology Applied
Scientific EffectPhotoelectric conversion: Photoelectric Effect

Data Source

PatentUS10573234B2Systems and methods for in-frame sensing and adaptive sensing control
Publication Date: 2020.02.25 APPLE INC
  • US10573234B2 patent drawing
  • US10573234B2 patent drawing
  • US10573234B2 patent drawing

AI summary

A method for operating an electronic display includes displaying image frames and receiving operational parameters of the electronic display based on illuminating a sense pixel of at least one row of pixels of the electronic display when displaying the image frames. A first set of pixels below the at least one row of pixels renders a portion of a first image frame and a second set of pixels above the at least one row of pixels renders a portion of a second image frame. The method also includes adjusting image display of a third image frame on the electronic display based on the operational parameters.