Hardware Overlay Assignment for Mobile Display Panels

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Mobile computing devices face performance issues due to the limited number of overlay windows in system on a chip (SoC) designs, leading to expensive pre-composition processes that consume memory bandwidth and power when handling multiple graphical layers, especially in applications like video streaming and gaming.

Innovation Solution

Implementing a display panel with embedded memory that supports partial updates, allowing the layer compositor to filter out static layers and only update actively changing regions, thereby reducing the number of overlays to be composited and minimizing memory bandwidth and power usage.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If the number of graphical layers exceeds the number of overlay windows, then all layers can be displayed, but pre-composition processes consume excessive memory bandwidth and power

Engineering Contradiction:

Improvenumber of graphical layersVSAvoidpower consumption

Core Design Contradiction:

Adaptability or versatilityVSUse of energy by moving object

Solution Approach 1:

The patent segments the display content into static layers and dynamic layers. Static layers (status bar, navigation bar, virtual keyboard) are separated from dynamic layers (video content, game graphics) and handled differently. This segmentation allows the system to avoid pre-composition for static layers while maintaining support for multiple dynamic layers, thereby reducing power consumption and memory bandwidth usage.

Inventive Principle:

Principle #1Segmentation

2Reliability

If pre-composition is performed for all layers, then overlay overflow is avoided, but memory bandwidth consumption increases significantly

Engineering Contradiction:

Improvedisplay correctnessVSAvoidmemory bandwidth

Core Design Contradiction:

ReliabilityVSQuantity of substance

Solution Approach 1:

The patent divides layers into static and dynamic categories. Static layers are identified and excluded from the pre-composition process, while only dynamic layers undergo pre-composition. This selective approach maintains display correctness for all layers while significantly reducing the amount of data that needs to be processed through memory, thereby reducing memory bandwidth consumption.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

Instead of performing pre-composition on all layers (excessive action), the patent applies pre-composition only to dynamic layers that actually require it (partial action). This partial application of the pre-composition process is sufficient to prevent overlay overflow issues while avoiding the unnecessary memory bandwidth consumption that would result from processing static layers.

Inventive Principle:

Principle #16Partial or excessive action

3Productivity

If dedicated hardware overlays are allocated to each application, then rendering performance improves, but the limited number of overlay windows causes overlay overflow

Engineering Contradiction:

Improverendering performanceVSAvoidnumber of overlay windows

Core Design Contradiction:

ProductivityVSAdaptability or versatility

Solution Approach 1:

The patent segments the layer management approach by distinguishing between static layers that can reuse overlay windows and dynamic layers that require dedicated overlays. This segmentation enables the system to maintain high rendering performance for dynamic content while efficiently reusing overlay windows for static content, thereby supporting more than the traditional limit of overlay windows.

Inventive Principle:

Principle #1Segmentation

4Adaptability or versatility

If frame buffers are used to store static content, then memory usage increases, but overlay windows are consumed

Engineering Contradiction:

Improveoverlay window availabilityVSAvoidmemory usage

Core Design Contradiction:

Adaptability or versatilityVSQuantity of substance

Solution Approach 1:

Instead of the traditional approach where static content is stored in frame buffers and consumes overlay windows, the patent inverts the approach by storing static content directly in the display panel's local memory and using overlay windows primarily for dynamic content. This inversion reduces both memory usage and overlay window consumption simultaneously.

Inventive Principle:

Principle #13The other way round (Inversion)

Data Source

PatentUS9881592B2Hardware overlay assignment

Publication Date: 2018.01.30 NVIDIA CORP

AI summary

An aspect of the present invention proposes a novel approach that can reduce the total number of the overlays to be composited during the display of graphical output in a mobile computing device. As a result, the total number of memory bandwidth and the usage of a graphics processing unit by a pre-compositor can be decreased significantly. According to one embodiment, this new approach is implemented with a display panel with embedded memory which supports a partial update, or refresh feature. Which such a feature, the layer compositor (typically either the display controller or GPU) is able to keep track of actively updating regions of a display panel by checking if each layer has new content to be displayed.