Display Subsystem Sequential Compositing for GPU Load Reduction

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Graphical user interfaces face challenges in efficiently utilizing hardware resources for window surface compositing when the number of application window surfaces exceeds the number of available hardware layers in a display subsystem, leading to increased load on the GPU and reduced performance.

Innovation Solution

The method involves using only the display subsystem to sequentially composite the final display frame, allowing the GPU to be available for rendering the next frame while the previous frame composition is ongoing, and utilizing an intermediate frame stored in memory to manage the composition process, enabling asynchronous composition and GPU offloading.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If the GPU is used to composite window surfaces, then the compositing quality is improved, but the GPU load increases and rendering performance deteriorates

Engineering Contradiction:
Improvecompositing qualityVSAvoidrendering performance
Core Design Contradiction:
ReliabilityVSProductivity

Solution Approach 1:

The patent segments the compositing task from the GPU by introducing a dedicated display subsystem with hardware layers. The GPU handles only rendering while the display subsystem handles compositing through sequential layer composition, thus separating these two functions to avoid GPU overload and maintain both compositing quality and rendering performance.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The patent introduces an intermediate frame buffer as a mediator between the GPU rendering output and the final display output. This intermediate frame allows the GPU to complete rendering independently while the display subsystem performs compositing operations separately, enabling asynchronous operation and preventing the GPU from being blocked by compositing tasks.

Inventive Principle:
Principle #24Intermediary (Mediator)

2Quantity of substance

If multiple hardware layers are used simultaneously, then the number of composited surfaces is increased, but the hardware resource complexity increases

Engineering Contradiction:
Improvenumber of composited surfacesVSAvoidhardware resource complexity
Core Design Contradiction:
Quantity of substanceVSDevice complexity

Solution Approach 1:

The patent implements periodic action by sequentially reusing the same hardware layers for compositing multiple window surfaces across different time periods. Instead of requiring simultaneous availability of multiple hardware layers, the system composites surfaces one after another using the same hardware resources, thus reducing hardware complexity while maintaining the ability to handle multiple surfaces.

Inventive Principle:
Principle #19Periodic action

Solution Approach 2:

The patent transitions from a spatial dimension approach (using multiple hardware layers simultaneously) to a temporal dimension approach (sequentially reusing hardware layers over time). This dimensional change allows the system to composite more surfaces than the number of physical hardware layers by utilizing time multiplexing, effectively increasing the number of composited surfaces without increasing hardware complexity.

Inventive Principle:
Principle #17Another dimension (Dimensionality change)

3Productivity

If the display subsystem is used for compositing, then the GPU load is reduced, but the compositing speed may be limited by hardware layer availability

Engineering Contradiction:
ImproveGPU availabilityVSAvoidcompositing speed
Core Design Contradiction:
ProductivityVSSpeed

Solution Approach 1:

The patent applies preliminary action by pre-configuring the display subsystem with a fixed set of hardware layers dedicated to compositing operations. This preliminary setup allows the display subsystem to independently handle compositing without requiring real-time GPU intervention, thus reducing GPU load while maintaining predictable compositing speeds through hardware-accelerated layer composition.

Inventive Principle:
Principle #10Preliminary action

Data Source

PatentEP3743802B1A method and an apparatus for display contents generation
Publication Date: 2024.01.24 HUAWEI TECH CO LTD
  • EP3743802B1 patent drawingFigure 1
  • EP3743802B1 patent drawingFigure 2
  • EP3743802B1 patent drawingFigure 3

AI summary

A method, an apparatus and a computer program for display contents generation are provided. The disclosed solution enables display contents generation in a case where several application window surfaces are visible at the same time and the number of application window surfaces exceeds the number of available hardware layers in a display subsystem (DSS). In the disclosed solution, the display subsystem is sequentially used to composite a final display frame.