In-Frame Sensing for Electronic Displays

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Electronic displays face challenges in sensing environmental and display-related operational parameters while maintaining image quality, particularly in dark environments where light emission during sensing can affect perceived image quality.

Innovation Solution

The electronic display performs sensing operations by illuminating sense pixels during the refresh of display pixels, allowing for simultaneous sensing and image frame refreshment without pausing the refresh process, thereby reducing perceivability of luminance variations.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If sensing operations are performed by illuminating sense pixels during image frame display, then environmental and display parameters can be sensed during normal operation, but light emission during sensing may affect perceived image quality especially in dark environments

Engineering Contradiction:
Improvesensing accuracyVSAvoidperceived image quality degradation
Core Design Contradiction:
Measurement precisionVSObject-affected harmful factors

Solution Approach 1:

The display panel is divided into two distinct pixel types: sense pixels dedicated to sensing operations and display pixels for image rendering. This segmentation allows sensing and display functions to coexist without interfering with each other, as sense pixels can be illuminated for sensing while display pixels maintain their intended visual output

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

Sense pixels act as intermediary elements between the sensing circuitry and the display output. These dedicated sense pixels capture environmental light and display parameter information without directly contributing to the visible image, serving as a mediator that enables sensing while preserving image quality

Inventive Principle:
Principle #24Intermediary (Mediator)

2Measurement precision

If the refresh process is paused during sensing operations, then sensing can be performed accurately, but image refresh rate and productivity are reduced

Engineering Contradiction:
Improvesensing accuracyVSAvoidrefresh rate
Core Design Contradiction:
Measurement precisionVSProductivity

Solution Approach 1:

The refresh process continues uninterrupted during sensing operations. Display pixels maintain their refresh cycle while sense pixels are simultaneously illuminated for sensing. This continuous operation of the refresh process ensures high productivity and maintains the intended refresh rate without pauses

Inventive Principle:
Principle #20Continuity of useful action

Solution Approach 2:

By segmenting the pixel array into sense pixels and display pixels, the system enables parallel operation where sensing and refreshing occur simultaneously on different pixel subsets, eliminating the need to pause refresh for sensing

Inventive Principle:
Principle #1Segmentation

3Productivity

If multiple non-contiguous refresh pixel groups are used to increase refresh rate, then productivity increases, but timing complexity and device complexity increase

Engineering Contradiction:
Improverefresh rateVSAvoidtiming complexity
Core Design Contradiction:
ProductivityVSDevice complexity

Solution Approach 1:

The sense pixel illumination scheme is designed to work universally with any refresh pixel group configuration. Whether using single or multiple non-contiguous refresh pixel groups, the same sense pixel illumination approach applies, making the sensing system adaptable to various refresh strategies without increasing its own complexity

Inventive Principle:
Principle #6Universality (Multi-functionality)

Applied Scientific Principles

This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.

Function Achieved in This Case

This approach enables the electronic display to accurately sense operational parameters while maintaining image quality and reducing the perceivability of sensing-related artifacts, ensuring consistent and high-quality image rendering.

Implementation Method 1

an electronic display may control light emission (e.g., actual luminance) from its display pixels

Methodology Applied
Scientific EffectLight emission from display pixels: Light Emitting Diode

Data Source

PatentUS10992911B2Systems and methods for in-frame sensing and adaptive sensing control
Publication Date: 2021.04.27 APPLE INC
  • US10992911B2 patent drawing
  • US10992911B2 patent drawing
  • US10992911B2 patent drawing

AI summary

A method for operating an electronic display includes receiving, using a controller, sensor data related to operational parameters of the electronic display based at least in part on illuminating a sense pixel of at least one row of pixels of the electronic display, wherein a first set of pixels below the at least one row of pixels renders a portion of a first image frame and a second set of pixels above the at least one row of pixels renders a portion of a second image frame. The method also includes adjusting, using the controller, image display on the electronic display based at least in part on the sensor data.