Conditional DMA Engine Using Bitmasks for Sparse Data

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing systems face inefficiencies in conditional direct memory access (DMA) operations, particularly in graph neural network procedures like BFS and page rank, due to wasted bandwidth and latency issues when dealing with sparse data, as they lack dedicated hardware solutions for manipulating graph data structures.

Innovation Solution

The implementation of instruction set architecture (ISA) and architectural support for conditional DMA operations, utilizing near-memory compute capabilities to efficiently perform gather, scatter, and broadcast operations based on user-provided bitmasks, reducing latency and bandwidth waste by moving only necessary data.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Speed

If standard systems pre-load data optimistically for conditional read/write operations, then data access speed is improved, but bandwidth is wasted when conditions are false

Engineering Contradiction:

Improvedata access speedVSAvoidbandwidth waste

Core Design Contradiction:

SpeedVSLoss of energy

Solution Approach 1:

The system performs preliminary actions by generating bitmasks ahead of time that indicate which data elements should be transferred, allowing the DMA operation to conditionally move only necessary data without wasting bandwidth on unnecessary transfers

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The invention changes the parameter of data transfer by using bitmasks to control which specific data elements are transferred during DMA operations, transforming a blanket transfer approach into a selective transfer approach that avoids bandwidth waste

Inventive Principle:

Principle #35Parameter changes

2Adaptability or versatility

If remote data is retrieved for conditional operations, then data availability is improved, but cost and latency increase significantly

Engineering Contradiction:

Improvedata availabilityVSAvoidlatency

Core Design Contradiction:

Adaptability or versatilityVSLoss of time

Solution Approach 1:

Bitmasks are generated in advance to identify which remote data elements need to be transferred, allowing the system to avoid retrieving remote data that won't be used, thereby reducing latency and cost while maintaining data availability for needed elements

Inventive Principle:

Principle #10Preliminary action

3Adaptability or versatility

If conditional data transfer is implemented without dedicated hardware support, then system flexibility is maintained, but performance and efficiency deteriorate

Engineering Contradiction:

Improvesystem flexibilityVSAvoidperformance efficiency

Core Design Contradiction:

Adaptability or versatilityVSProductivity

Solution Approach 1:

The DMA engine is enhanced with multi-functionality to support conditional data transfer operations using bitmasks, allowing a single hardware component to handle both traditional DMA operations and conditionalgather/scatter operations, maintaining flexibility while improving performance

Inventive Principle:

Principle #6Universality (Multi-functionality)

Data Source

PatentUS20230333998A1Instruction set architecture support for conditional direct memory access data movement operations

Publication Date: 2023.10.19 INTEL CORP

AI summary

Systems, apparatuses and methods may provide for technology that includes a plurality of memory engines corresponding to a plurality of pipelines, wherein each memory engine in the plurality of memory engines is adjacent to a pipeline in the plurality of pipelines, and wherein a first memory engine is to request one or more direct memory access (DMA) operations associated with a first pipeline, and a plurality of operation engines corresponding to a plurality of dynamic random access memories (DRAMs), wherein each operation engine in the plurality of operation engines is adjacent to a DRAM in the plurality of DRAMs, and wherein one or more of the plurality of operation engines is to conduct the one or more DMA operations based on one or more bitmaps.