Structured Document Scrolling with Bounded Memory via GPU Offloading
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Rendering large structured documents with significant media content in graphical user interfaces can lead to performance issues such as choppy scrolling and frozen screens due to high hardware resource consumption, particularly when rendering media data constantly swaps in and out of main memory.
Innovation Solution
The method involves using graphics processors to accelerate the rendering of media content while main processors handle text and frame elements, determining rendering regions based on user behavior, social factors, or user preferences, and optimizing memory usage through techniques like bin-packing and ant-hill optimization to improve scrolling performance and reduce hardware burden.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If media content is rendered using main processors, then rendering can be performed, but hardware resource consumption increases and scrolling performance deteriorates
Solution Approach 1:
The patent segments the rendering workload by separating media content rendering from text and frame element rendering. Graphics processors handle media content (images, videos, animations) while main processors handle text and layout elements. This division allows parallel processing and prevents the main processor from being overwhelmed by media rendering tasks, thus maintaining smooth scrolling while achieving adequate rendering performance.
Solution Approach 2:
The patent introduces a layout engine as an intermediary component that coordinates between main processors and graphics processors. The layout engine determines which elements should be rendered by which processor based on element types, user behavior predictions, and available resources. This intermediary manages the complex interaction between different processing units, optimizing the overall rendering pipeline without requiring direct communication between main and graphics processors.
2Ease of operation
If all media content is rendered in advance to improve scrolling performance, then scrolling smoothness improves, but memory consumption increases
Solution Approach 1:
The patent applies preliminary action by pre-rendering only the media content that is likely to be visible based on predicted user scrolling behavior. The system analyzes user interaction patterns, document structure, and media importance to determine which media elements should be rendered in advance. This selective pre-rendering approach smooths scrolling performance for anticipated content while avoiding the memory overhead of pre-rendering the entire document.
Solution Approach 2:
The patent dynamically adjusts rendering parameters based on available memory resources and observed user behavior. When memory is constrained, the system reduces the pre-rendering window and focuses on high-priority media elements. When memory is abundant, it expands the pre-rendering scope. This parameter adaptation allows the system to maintain scrolling smoothness across different device configurations without excessive memory consumption.
Data Source
AI summary
In one embodiment, one or more main processors of a device detect an event corresponding to scrolling a structured document presented in a scrollable graphical user interface of an application executed by the one or more main processors. The main processors, in response to the event, determine a first set of elements and a second set of elements of the structured document; render the first set of elements of the structured document for presentation in the graphical user interface; and cause one or more graphics processors of the device to render the second set of elements for presentation in the graphical user interface.