GPU Coherency Manager for Memory Request Routing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In multi-processor systems, accessing GPU local memory is often non-coherent, requiring software intervention to ensure coherency, which leads to performance overhead.
Innovation Solution
Implementing coherent memory in GPU local memory, allowing hardware to ensure coherency without software overhead, and using probe engines and filters to manage cache coherence between CPU and GPU systems.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If software is used to ensure memory coherency between CPU and GPU, then coherency is maintained, but performance overhead increases
Solution Approach 1:
The patent replaces software-based coherency management with a hardware-based solution. A coherency manager circuit is implemented in the GPU that automatically handles memory coherency by intercepting memory requests, determining their origin (CPU or GPU), and routing them appropriately through the interconnect to the system memory or local memory, eliminating the need for software intervention
Solution Approach 2:
The patent introduces a coherency manager as an intermediary component between the GPU and memory systems. This manager acts as a mediator that receives memory requests from the GPU, determines whether they are coherent or non-coherent, and routes them to the appropriate memory (system memory or local memory), thereby automatically maintaining coherency without software overhead
2Ease of operation
If GPU local memory is made coherent with system memory, then programming is simplified, but hardware complexity increases
Solution Approach 1:
The patent segments memory into two distinct types: coherent memory (system memory accessible by both CPU and GPU) and non-coherent memory (local memory private to GPU). The coherency manager segments memory requests into coherent requests (routed to system memory) and non-coherent requests (routed to local memory), allowing programmers to use a unified memory model while the hardware handles the complexity of segmentation internally
Solution Approach 2:
The coherency manager serves as an intermediary that abstracts the underlying hardware complexity from programmers. It automatically determines whether memory requests are coherent or non-coherent and routes them appropriately, allowing programmers to write code as if all memory is coherent without manually managing the complexity of hardware segmentation
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A processing system is provided. The processing system includes a first processing unit coupled to a first memory and a second processing unit coupled to a second memory. The second memory comprises a coherent memory and a private memory that is private to the second processing unit.