Dynamic Processing-in-Memory Caching for Lower PIM-Load Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional computer architectures suffer from increased data transfer latency and memory access energy due to remote processing units, which can be mitigated by processing-in-memory (PIM) architectures. However, PIM-Load requests in these architectures trigger multiple lookups and increase contention for shared resources, leading to performance degradation.
Innovation Solution
Dynamic caching policies are implemented in PIM architectures, assigning different modes of operation to memory pages based on the execution phase, such as host, hybrid, transition, and PIM modes, to optimize data access and reduce latency and resource contention.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If PIM-Load requests are processed using conventional cache coherence protocols, then data coherence between caches and memory is maintained, but multiple lookups and resource contention increase latency
Solution Approach 1:
The system performs preliminary actions by flushing dirty cache lines to memory before PIM-Load operations and pre-establishing memory-only access modes for PIM workloads. This ensures data coherence is maintained in advance, eliminating the need for multiple lookups during actual PIM-Load operations and reducing latency.
Solution Approach 2:
The system dynamically switches caching policies based on workload phase. During PIM execution phases, it transitions to memory-only access mode with disabled caches, while enabling caches during host processor phases. This dynamic adaptation resolves the contradiction by optimizing for latency during PIM operations while maintaining coherence during host operations.
2Speed
If caches are used to store frequently accessed data, then host processor access speed is improved, but PIM units experience increased latency due to coherence protocol overhead
Solution Approach 1:
The system dynamically configures caching behavior based on the executing workload. When host processor executes workloads, caches are enabled with write-back policies to improve host access speed. When PIM units execute workloads, caches are disabled or configured for memory-only access to eliminate coherence overhead and reduce PIM access latency.
Solution Approach 2:
The system creates separate access pathways for host and PIM operations. Host processors access data through the cache hierarchy, while PIM units access data directly from memory. This copying of the data access system allows each processor type to use the optimal path, resolving the speed-latency contradiction.
3Use of energy by moving object
If write-back caching policy is used to reduce memory writes, then energy consumption is reduced, but data coherence maintenance becomes more complex
Solution Approach 1:
The system performs preliminary flush operations to write dirty cache lines to memory before PIM workload execution. This preliminary action ensures memory contains up-to-date data, simplifying coherence management during PIM operations while still allowing write-back caching to reduce energy consumption during host operations.
Solution Approach 2:
The system applies different caching policies to different memory regions or access patterns. Write-back caching is used for regions accessed by host processors to reduce energy consumption, while memory-only access is used for regions accessed by PIM units to simplify coherence. This local differentiation resolves the energy-complexity contradiction.
Data Source
AI summary
A computing device includes a memory having one or more processing-in-memory units, and a host processor that includes one or more caches. The host processor receives a workload that accesses a page of the memory. Further, the host processor dynamically assigns different modes of operation to the page of the memory during different phases of execution of the workload. The different modes are assigned based on whether the workload is to be executed by the host processor, the one or more processing-in-memory units, or the host processor and the one or more processing-in-memory units during the different phases of execution. The host processor processes requests of the workload that access the page of the memory in accordance with the different modes of operation during the different phases of execution. Notably, the different modes of operation specify different caching policies for the page of the memory.