Multi-Chiplet GPU Task Scheduling for Local Memory Access
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing parallel processors, such as GPUs, face inefficiencies in task and data assignment due to the high latency and energy consumption associated with accessing off-chiplet data, which hinders their performance in executing compute shaders and machine learning applications.
Innovation Solution
Implementing a multi-chiplet processor with advanced processing chiplets (APCs) that share a single pool of virtual and physical memory, and using a scheduler to optimize task and data assignment to minimize off-chiplet data access by interleaving data across associated high-bandwidth memories and assigning tasks in a manner that matches the data assignment order, thereby reducing latency and energy consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If data is stored in off-chiplet memory, then memory capacity is increased, but access latency and energy consumption increase
Solution Approach 1:
The system is divided into multiple chiplets, each with its own local memory (on-chiplet memory). Data is segmented and distributed across these local memories. When a task needs data, it first accesses the local memory of the same chiplet, avoiding off-chiplet access. This segmentation allows the system to maintain high memory capacity while reducing access latency for frequently used data.
Solution Approach 2:
The patent implements a hierarchical memory structure where small, fast on-chiplet memories are nested within each chiplet, and these are combined to form a larger off-chiplet memory pool. The hierarchy allows the system to provide both large total memory capacity and fast local access by nesting fast small memories within slower large memories.
2Quantity of substance
If data is stored in off-chiplet memory, then memory capacity is increased, but energy consumption increases
Solution Approach 1:
Each chiplet is equipped with local memory resources that are optimized for fast, low-energy access. The system assigns tasks to chiplets based on data locality, ensuring that data is accessed from the nearest possible memory location. This local quality optimization reduces the energy required for data transfer by minimizing off-chiplet memory accesses.
Solution Approach 2:
The scheduler performs preliminary assignment of tasks to chiplets based on where the required data is stored. By predicting data access patterns and pre-assigning tasks to the appropriate chiplets, the system avoids unnecessary off-chiplet data transfers, thereby reducing energy consumption before the actual computation begins.
3Ease of operation
If tasks are assigned without considering data location, then task scheduling is simplified, but data access efficiency decreases
Solution Approach 1:
The scheduler incorporates feedback about data location and access patterns when assigning tasks to chiplets. By monitoring which data is accessed most frequently and where it is stored, the scheduler dynamically adjusts task assignments to maximize data locality. This feedback mechanism maintains relatively simple scheduling logic while significantly improving data access efficiency.
Solution Approach 2:
The system changes the scheduling parameter from simple round-robin or static assignment to dynamic assignment based on data location parameters. The scheduler considers parameters such as data residence location, access frequency, and chiplet utilization to determine optimal task-chiplet mappings, thereby improving data access efficiency without overly complicating the scheduling mechanism.
Data Source
AI summary
Efficient task and data assignment is provided in multi-chiplet processors including one or more advanced processing chiplets (APCs). A graphics processing unit (GPU) assigns data for use by one or more tasks to memories associated with a plurality of APCs and one or more CPCs. A scheduler or other controller within or otherwise associated with the GPU assigns tasks, which utilize the assigned data, to the APCs. The GPU ensures efficient data assignment by adjustably interleaving data across memories associated with the APCs in order to limit off-chiplet remote memory traffic. Similarly, the scheduler ensures efficient task assignment by adjustably assigning tasks to the APCs, typically in the same order as or in a similar order to the placement order in which the data is assigned to the memories, in order to limit off-chiplet remote memory traffic.