CPU-GPU Data Transfer via Shared Cache Hierarchy
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The existing data transfer mechanism between a CPU and a GPU in a producer-consumer model is inefficient due to low latency and low bandwidth, as it relies on non-temporal stores that write data directly to main memory, resulting in weakly ordered operations and the need for additional Fence instructions to enforce ordering.
Innovation Solution
The proposed solution involves using the highest common cache level shared by both CPU and GPU for data exchange, writing data to write-combining buffers and then moving it through the cache hierarchy without writing to main memory, using instructions like MovNonAllocate to facilitate efficient data transfer and employing a flag to indicate readiness, allowing the GPU to read data from the cache.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If non-temporal stores are used to transfer data from CPU to GPU, then data can be written directly to main memory, but the transfer latency increases and bandwidth decreases
Solution Approach 1:
The patent introduces a cache memory hierarchy (L1, L2, L3 caches) as an intermediary between the CPU and main memory. Data is first stored in CPU caches using non-temporal store instructions, then transferred through the cache hierarchy to the GPU, avoiding direct CPU-to-main-memory transfers. This intermediary cache system reduces transfer latency and increases bandwidth by utilizing faster cache memory and parallel transfer paths.
Solution Approach 2:
The patent adds a temporal dimension to data transfer by using write-combining buffers that accumulate data over time before transfer. Instead of immediate direct transfers, data is buffered and combined across multiple write operations, then transferred in larger chunks through the cache hierarchy. This temporal buffering approach improves bandwidth utilization and reduces overall transfer time.
2Productivity
If non-temporal stores are used for data transfer, then data can be written to main memory, but memory ordering becomes weak and requires additional Fence instructions
Solution Approach 1:
The cache memory hierarchy acts as an intermediary that provides strong ordering guarantees. By transferring data through the cache system rather than directly to main memory, the patent maintains memory ordering semantics without requiring additional Fence instructions. The cache coherence protocol ensures proper ordering of memory operations, simplifying the instruction sequence while maintaining data transfer efficiency.
3Productivity
If data is transferred through the cache hierarchy instead of directly to main memory, then transfer efficiency improves, but cache management complexity increases
Solution Approach 1:
The patent employs self-service mechanisms where the cache coherence protocol automatically manages data consistency and ordering. The hardware infrastructure includes dedicated coherence controllers and buffer management logic that autonomously handle cache state transitions, invalidation, and data transfer coordination. This self-managing approach reduces the burden on software and simplifies cache management despite the increased hardware complexity.
Data Source
AI summary
An apparatus and method are described for efficiently transferring data from a core of a central processing unit (CPU) to a graphics processing unit (GPU). For example, one embodiment of a method comprises: writing data to a buffer within the core of the CPU until a designated amount of data has been written; upon detecting that the designated amount of data has been written, responsively generating an eviction cycle, the eviction cycle causing the data to be transferred from the buffer to a cache accessible by both the core and the GPU; setting an indication to indicate to the GPU that data is available in the cache; and upon the GPU detecting the indication, providing the data to the GPU from the cache upon receipt of a read signal from the GPU.