GPU Vertex Buffer Management for Higher Task Concurrency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The efficiency of graphics processing units (GPUs) is hindered by the need to pause tasks due to memory latency, which is exacerbated by the limited availability of on-chip storage for geometry data, leading to a cap on the number of tasks that can execute concurrently.
Innovation Solution
Allocate off-chip storage to geometry tasks upon creation and manage on-chip storage allocation dynamically based on availability, allowing geometry data to be written to either on-chip or off-chip storage as needed, thereby decoupling the number of tasks from the size of on-chip storage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If on-chip storage is allocated to geometry tasks upon creation, then the number of concurrent tasks is limited by available on-chip storage, but memory latency can be reduced by having data ready in fast storage
Solution Approach 1:
The patent introduces a hierarchical storage architecture that adds a temporal dimension to storage allocation. Instead of allocating on-chip storage upfront for all tasks, the system uses off-chip storage for initial allocation and dynamically transfers data to on-chip storage when needed, effectively adding a time-based layer to the storage hierarchy that decouples spatial constraints from task concurrency.
Solution Approach 2:
The system performs preliminary allocation of off-chip storage to geometry tasks upon creation, ensuring data can be prepared in advance. This preliminary action allows tasks to be created and queued without immediately consuming on-chip resources, while still enabling latency hiding by having data ready in off-chip storage before execution.
2Productivity
If on-chip storage size is increased to accommodate more concurrent tasks, then task concurrency improves, but physical GPU size and cost increase
Solution Approach 1:
The patent segments the storage system into off-chip and on-chip portions, with the off-chip storage serving as a large-capacity reservoir and the on-chip storage serving as a high-speed workspace. This segmentation allows the system to achieve high task concurrency through off-chip storage while keeping on-chip storage small, thereby maintaining compact physical GPU size.
Solution Approach 2:
The patent introduces a storage management unit as an intermediary that mediates between off-chip storage and geometry processing units. This intermediary dynamically manages data transfer between storage hierarchies, enabling the system to support many concurrent tasks using affordable off-chip storage without requiring large amounts of expensive on-chip storage.
3Productivity
If dynamic on-chip storage allocation is implemented, then storage utilization efficiency improves, but allocation complexity and management overhead increase
Solution Approach 1:
The storage management unit implements self-service mechanisms by automatically monitoring on-chip storage availability and dynamically allocating or deallocating regions based on real-time conditions. This self-service approach improves storage utilization efficiency while minimizing the need for complex external management intervention.
Solution Approach 2:
The system incorporates feedback mechanisms where the storage management unit continuously monitors on-chip storage usage and adjusts allocations dynamically. This feedback loop enables efficient storage utilization by responding to actual usage patterns while keeping management complexity manageable through automated decision-making based on predefined criteria.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A method of managing resources in a GPU is described. The method comprises allocating a region of off-chip storage to a geometry task on creation of the geometry task and receiving, at an on-chip store in the GPU, a memory allocation request for the geometry task from a shader core in the GPU, wherein the memory allocation request is received after generation of geometry data for the geometry task. In response to receiving the memory allocation request, the method comprises determining, by the on-chip store, whether to allocate a region of the on-chip store to the geometry task. In response to allocating the region of the on-chip store, geometry data for the geometry task is written to the on-chip store and in response to determining not to allocate the region of the on-chip store, the geometry data is written to the allocated region of off-chip storage.