GPU Data Multicasting for Lower L2 Cache Bandwidth Demand
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing GPU architectures face challenges in achieving strong scaling by reducing tile sizes, which leads to increased demands on L2 cache bandwidth and memory fetch redundancy, resulting in inefficient use of resources and power consumption.
Innovation Solution
Implementing a programmatic multicast mechanism that allows coordinated data fetching by a processor on behalf of multiple processors, leveraging data fetch redundancy to reduce bandwidth and power requirements, and enhancing L2 cache efficiency through multicast of response data to multiple cooperative thread arrays.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If tile sizes are reduced to achieve strong scaling, then processing speed for single applications is improved, but L2 cache bandwidth demand increases and memory fetch redundancy occurs
Solution Approach 1:
The patent merges multiple independent memory fetch operations into a single coordinated fetch operation. When multiple processing units (e.g., thread blocks) need the same data from L2 cache, the system combines these requests into one unified fetch operation, reducing redundant memory accesses and lowering L2 cache bandwidth demand while maintaining high processing speed.
2Speed
If tile sizes are reduced to achieve strong scaling, then processing speed for single applications is improved, but memory fetch redundancy increases
Solution Approach 1:
Instead of having multiple processing units independently copy the same data from memory (which causes redundancy), the system uses a coordinated approach where one processing unit fetches the data and makes it available to others through shared memory or cache mechanisms, eliminating redundant copy operations and reducing power consumption.
3Use of energy by moving object
If coordinated data fetching is implemented, then bandwidth and power requirements are reduced, but system complexity increases
Solution Approach 1:
The patent introduces intermediary structures such as shared memory buffers or cache coordination mechanisms that mediate between multiple processing units and the L2 cache. These intermediaries manage the coordination of fetch operations, tracking which data has been fetched and distributing it to requesting units, thereby reducing bandwidth consumption while managing system complexity through dedicated coordination layers.
Data Source
AI summary
This specification describes a programmatic multicast technique enabling one thread (for example, in a cooperative group array (CGA) on a GPU) to request data on behalf of one or more other threads (for example, executing on respective processor cores of the GPU). The multicast is supported by tracking circuitry that interfaces between multicast requests received from processor cores and the available memory. The multicast is designed to reduce cache (for example, layer 2 cache) bandwidth utilization enabling strong scaling and smaller tile sizes.