Decoupled Access-Execute Near-Memory Processing for Bandwidth Efficiency

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing near-memory processing (NMP) architectures face limitations in die area, power budget, and logic complexity, leading to suboptimal memory bandwidth utilization and high memory access latencies, especially in von-Neumann style, many-core processors.

Innovation Solution

Implement a decoupled access-execute architecture (DAE-NMP) with specialized access processors (APs) and execute processors (EPs) to separate data movement and computation tasks, minimizing resource overprovisioning and enhancing memory bandwidth utilization and latency tolerance.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If von-Neumann style many-core processors are used for near-memory processing, then memory bandwidth utilization is improved, but die area and power consumption increase due to resource overprovisioning

Engineering Contradiction:

Improvememory bandwidth utilizationVSAvoiddie area

Core Design Contradiction:

ProductivityVSArea of stationary object

Solution Approach 1:

The processor is segmented into two specialized units: access processors (APs) dedicated to memory access operations and execute processors (EPs) dedicated to computation. This segmentation eliminates the need for each core to be overprovisioned with both access and execute capabilities, reducing die area while maintaining high memory bandwidth utilization through dedicated APs

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

Different parts of the system are given different functional qualities: APs are optimized for memory access operations (load/store, address calculation) while EPs are optimized for compute operations. This local specialization allows each component to be minimized in size for its specific function, reducing overall die area compared to general-purpose many-core processors

Inventive Principle:

Principle #3Local quality

2Productivity

If von-Neumann style many-core processors are used for near-memory processing, then memory bandwidth utilization is improved, but power consumption increases due to resource overprovisioning

Engineering Contradiction:

Improvememory bandwidth utilizationVSAvoidpower consumption

Core Design Contradiction:

ProductivityVSUse of energy by stationary object

Solution Approach 1:

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

Inventive Principle:

Principle #3Local quality

3Area of stationary object

If simple processing elements are situated on the memory side for near-memory processing, then die area is reduced, but memory access latencies remain high due to limited processing capability

Engineering Contradiction:

Improvedie areaVSAvoidmemory access latency

Core Design Contradiction:

Area of stationary objectVSLoss of time

Solution Approach 1:

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The decoupled architecture enables continuous operation where APs continuously prepare data and EPs continuously process it. Multiple APs can service multiple EPs simultaneously, ensuring that compute operations never wait idle for data preparation, thereby reducing effective memory access latency through pipelined continuous operation

Inventive Principle:

Principle #20Continuity of useful action

4Use of energy by stationary object

If domain-specific accelerators are used in NMP architecture, then power efficiency is improved, but adaptability and programmability are reduced

Engineering Contradiction:

Improvepower efficiencyVSAvoidprogrammability

Core Design Contradiction:

Use of energy by stationary objectVSAdaptability or versatility

Solution Approach 1:

While APs and EPs are specialized for specific functions, the architecture maintains universality through configurable operation modes and the ability to handle various data types and workloads. The system can be programmed to perform different access patterns and compute operations, providing adaptability without requiring full general-purpose cores, thus balancing power efficiency with programmability

Inventive Principle:

Principle #6Universality (Multi-functionality)

Data Source

PatentUS12393421B2Techniques for decoupled access-execute near-memory processing

Publication Date: 2025.08.19 INTEL CORP

AI summary

Techniques for decoupled access-execute near-memory processing include examples of first or second circuitry of a near-memory processor receiving instructions that cause the first circuitry to implement system memory access operations to access one or more data chunks and the second circuitry to implement compute operations using the one or more data chunks.