Decoupled Access-Execute Near-Memory Processing for Bandwidth Utilization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing near-memory processing (NMP) architectures face limitations in die area, power budget, and logic complexity, leading to suboptimal memory bandwidth utilization and increased latency due to overprovisioning of cores responsible for both data access and computation tasks.
Innovation Solution
Implement a decoupled access-execute architecture (DAE-NMP) with specialized access processors (APs) and execute processors (EPs) to separate data movement and computation tasks, minimizing resource overprovisioning and enhancing memory bandwidth utilization and latency tolerance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If von-Neumann style many-core processors are used in NMP architectures, then memory bandwidth utilization is improved, but die area and power consumption increase due to overprovisioning
Solution Approach 1:
The processor is segmented into two distinct functional units: access processors (APs) dedicated to memory access operations and execute processors (EPs) dedicated to computation. This segmentation allows each unit to be optimized for its specific function, reducing the need for overprovisioning resources that would be required if single cores had to handle both access and execute tasks
Solution Approach 2:
Multiple access processors and execute processors are combined to work in parallel on memory-bound and compute-bound operations respectively. This merging of specialized units achieves high memory bandwidth utilization without requiring each individual core to be overprovisioned with full capabilities for both access and execution
2Productivity
If von-Neumann style many-core processors are used in NMP architectures, then memory bandwidth utilization is improved, but power consumption increases due to overprovisioning
Solution Approach 1:
The power consumption is reduced through segmentation by separating access and execute functions into dedicated units. Access processors consume power optimized for memory control operations while execute processors consume power optimized for computational operations, eliminating the need to power full-capability cores for tasks they don't primarily perform
Solution Approach 2:
Different parts of the processing system have different power characteristics matched to their function: access processors have power profiles suited for memory interface control while execute processors have power profiles suited for arithmetic operations, achieving efficient overall power utilization
3Device complexity
If single cores handle both data access and computation tasks, then device complexity is reduced, but latency increases due to interleaved operations
Solution Approach 1:
By segmenting the processing functions into separate access and execute units, the system eliminates the latency caused by interleaved operations. Access processors can continuously manage memory transactions while execute processors continuously perform computations, with both operating in parallel without interfering with each other's timing
Solution Approach 2:
A coordination mechanism acts as an intermediary between access processors and execute processors, managing data flow and synchronization. This intermediary enables the two specialized units to work in parallel efficiently, achieving low latency without requiring each core to independently manage both access and execution
Data Source
AI summary
Techniques for decoupled access-execute near-memory processing include examples of first or second circuitry of a near-memory processor receiving instructions that cause the first circuitry to implement system memory access operations to access one or more data chunks and the second circuitry to implement compute operations using the one or more data chunks.