GPU Warp Execution via Scratchpad Memory Status Indicators
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The limited scratchpad memory shared among streaming processors in GPUs becomes a bottleneck, restricting the number of warps that can be scheduled and executed efficiently.
Innovation Solution
A warp execution method that dynamically checks and manages the status of the scratchpad memory using indicators to determine when to load or compute warps, optimizing space usage by identifying blank, data-in-use, data-not-in-use, and data-loading spaces, and sending commands to the load/store engine to manage resource allocation effectively.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If the scratchpad memory size is increased to accommodate more warps, then the number of warps that can be scheduled increases, but the device complexity and cost increase
Solution Approach 1:
The scratchpad memory is divided into multiple status regions (blank, data-in-use, data-not-in-use, data-loading) with corresponding indicators. This segmentation allows the system to track and manage memory space efficiently by identifying which regions are available for different operations, thereby increasing warp scheduling capacity without physically expanding the memory device.
Solution Approach 2:
The patent introduces dynamic status indicators that change based on real-time memory usage conditions. The indicators (first indicator for data-in-use/blank boundary, second indicator for data-not-in-use/data-loading boundary, third indicator for data-in-use/data-not-in-use boundary) dynamically reflect the current state of the scratchpad memory, enabling flexible warp scheduling decisions that maximize utilization without requiring additional hardware.
2Adaptability or versatility
If the scratchpad memory is shared by multiple SPs, then resource utilization improves, but the limited space becomes a bottleneck for the number of warps
Solution Approach 1:
The patent implements a feedback mechanism through status indicators that continuously monitor the scratchpad memory state. The first indicator tracks the boundary between blank and data-in-use regions, the second indicator tracks the boundary between data-not-in-use and data-loading regions, and the third indicator tracks the boundary between data-in-use and data-not-in-use regions. This feedback enables SPs to make informed scheduling decisions about when to load or compute warps, maximizing the utilization of shared memory resources across multiple SPs.
Solution Approach 2:
The system changes the parameter of memory space utilization by introducing multiple status categories and corresponding indicators. Instead of treating scratchpad memory as a single homogeneous resource, the system divides it into distinct status regions with different characteristics (blank, data-in-use, data-not-in-use, data-loading). This parameter change allows for more nuanced management of shared memory, enabling multiple SPs to schedule warps more effectively without the limited space becoming a bottleneck.
3Productivity
If warp loading and computation are performed continuously, then processing throughput increases, but memory space management becomes more complex
Solution Approach 1:
The scratchpad memory is segmented into four distinct status regions: blank, data-in-use, data-not-in-use, and data-loading. Each region is tracked by specific indicators (first indicator for blank/data-in-use boundary, second indicator for data-not-in-use/data-loading boundary, third indicator for data-in-use/data-not-in-use boundary). This segmentation simplifies memory space management during continuous operations by providing clear, discrete information about available and occupied regions, enabling SPs to efficiently decide when to load or compute warps without managing complex memory states.
Data Source
AI summary
The present application discloses a warp execution method used for SPs of an SM of a GPU and an associated GPU. The SPs share a scratchpad memory, and the warp execution method includes: when the predetermined time point for warp-loading is reached, checking a first indicator to obtain a size of a space with the status of blank in the scratchpad memory, to determining whether to load the warp, wherein the first indicator is used to indicate a starting position of a space with the status of data-in-use and an ending position of the space with the status of blank; and when the predetermined time point for computing is reached, checking a second indicator and a third indicator to obtain a size of a space with the status of data-not-in-use in the scratchpad memory, to determining whether to compute the warp.