Runtime-Assisted IMC/NMC/FMC Co-Processing for Graph Analytics

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Graph analytics systems face performance degradation due to excessive data movement, redundant data transfer, and inefficient resource utilization, particularly in large-scale graph processing systems with complex compute parallelism and memory access patterns, leading to increased network traffic, compute resource under-utilization, and energy consumption.

Innovation Solution

Implement a memory hierarchy with in-memory compute (IMC), near-memory compute (NMC), and far-memory compute (FMC) elements, along with software runtime directed refactoring, to process data where it resides, reducing unnecessary data movement by filtering out unneeded data and utilizing compiler directives to map instructions to appropriate compute elements.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If data is moved from memory to compute registers through cache hierarchy, then data processing can be performed, but excessive data movement overhead and energy consumption occur

Engineering Contradiction:

Improvedata processing throughputVSAvoidenergy consumption

Core Design Contradiction:

ProductivityVSLoss of energy

Solution Approach 1:

The patent segments the memory hierarchy into multiple levels (L1, L2, L3 caches and main memory) with compute elements distributed at different levels. This segmentation allows data to be processed at the closest compute element without moving the entire dataset to the main processor, reducing energy consumption while maintaining processing throughput.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent introduces intermediate compute elements (such as cache-attached processors or near-memory compute units) that act as mediators between main memory and the main processor. These intermediaries perform preliminary data processing and filtering, reducing the volume of data that needs to be transferred to the main processor, thus lowering energy consumption while preserving productivity.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Productivity

If large amounts of data are moved from memory to compute nodes, then graph analytics can be performed, but network traffic and data movement overhead increase

Engineering Contradiction:

Improvegraph analytics processing speedVSAvoiddata movement volume

Core Design Contradiction:

ProductivityVSQuantity of substance

Solution Approach 1:

The patent implements local compute elements at different levels of the memory hierarchy that perform graph analytics operations locally on data before it is moved to higher levels. This local processing reduces the volume of data that needs to be transferred across the network and memory hierarchy, while maintaining analytical processing speed through distributed computation.

Inventive Principle:

Principle #3Local quality

Solution Approach 2:

The patent applies preliminary filtering and processing operations at lower levels of the memory hierarchy before data is moved to higher levels or to compute nodes. This preliminary action reduces the volume of data that needs to be transferred, decreasing network traffic and data movement overhead while preserving the ability to perform comprehensive graph analytics.

Inventive Principle:

Principle #10Preliminary action

3Loss of energy

If compute elements process data at different levels of memory hierarchy, then data movement is reduced, but system complexity increases

Engineering Contradiction:

Improveenergy efficiencyVSAvoidmemory hierarchy structure

Core Design Contradiction:

Loss of energyVSDevice complexity

Solution Approach 1:

The patent designs compute elements with multi-functionality that can operate at different levels of the memory hierarchy and perform various operations (filtering, aggregation, graph analytics). This universality reduces the need for specialized hardware at each level, managing system complexity while maintaining energy efficiency benefits of localized processing.

Inventive Principle:

Principle #6Universality (Multi-functionality)

Solution Approach 2:

The patent implements a dynamic memory hierarchy where compute elements can be activated or deactivated based on workload requirements, and data can be placed at different hierarchy levels based on access patterns. This dynamic approach allows the system to adapt to different workloads, managing complexity while preserving energy efficiency through intelligent resource allocation.

Inventive Principle:

Principle #15Dynamics

Data Source

PatentUS20250307180A1Software Runtime Assisted Co-Processing Acceleration with a Memory Hierarchy Augmented with Compute Elements

Publication Date: 2025.10.02 ADVANCED MICRO DEVICES INC

AI summary

The concepts and technologies disclosed herein are directed to software runtime assisted co-processing acceleration with a memory hierarchy augmented with compute elements. An example system disclosed herein includes one or more switches and a plurality of hardware compute nodes connected via the one or more switches. Each hardware compute node of the plurality of hardware compute nodes includes an in-memory compute (IMC) element configured to perform in-memory processing operations on data, such as graph data. The system also includes a near-memory compute (NMC) element configured to perform near-memory processing operations on the data. The system also includes a far-memory compute (FMC) element configured to perform far-memory processing operations on the data.