DMA-DRAM Re-Order Buffer and Interleaving for NPU Bottlenecks

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

The bottleneck phenomenon in data transfer between a DMA and DRAM in SoCs, particularly when performing neural network operations, is addressed by reducing data transfer speed and latency through a connection device using a re-order buffer and interleaving.

Innovation Solution

A connection device with a slave interface, master interface, and switch is employed to manage data read requests from a DMA to a DRAM, utilizing a re-order buffer to rearrange data order and memory interleaving to optimize data transfer.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If data transfer is performed directly between DMA and DRAM, then data transfer speed is maintained at baseline level, but bottleneck phenomenon occurs during neural network operations

Engineering Contradiction:

Improveoperation processing speedVSAvoiddata transfer latency

Core Design Contradiction:

ProductivityVSLoss of time

Solution Approach 1:

The connection device performs preliminary actions by receiving and buffering data from DRAM before the NPU actually needs it for neural network operations. The re-order buffer stores data in advance, allowing the NPU to retrieve data immediately when needed, thus reducing operational latency and improving processing speed.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The connection device acts as an intermediary component between the DRAM and the NPU's DMA. It includes a re-order buffer that temporarily holds data and a scheduling unit that manages data flow, mediating the data transfer process to eliminate bottlenecks and optimize the interaction between memory and processing units.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Speed

If data is read sequentially from DRAM, then data order is maintained, but data transfer speed decreases due to bottleneck at connector

Engineering Contradiction:

Improvedata transfer speedVSAvoiddata order consistency

Core Design Contradiction:

SpeedVSStability of the object's composition

Solution Approach 1:

The connection device segments the data stream by dividing incoming data from DRAM into multiple channels or queues within the re-order buffer. The scheduling unit can then manage these segments independently, allowing parallel processing and faster data transfer while maintaining the ability to reassemble them in the correct order for the NPU.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The connection device dynamically adjusts data flow management based on operational needs. The scheduling unit can change data transfer priorities, buffer allocation, and routing paths in real-time, optimizing data transfer speed while ensuring correct data ordering is maintained through intelligent dynamic control rather than fixed sequential access.

Inventive Principle:

Principle #15Dynamics

Data Source

PatentUS12487946B2Connection device between DMA and DRAM using re-order buffer and interleaving and method of using the same

Publication Date: 2025.12.02 MOBILINT INC

AI summary

Disclosed is a connection device between a DMA and a DRAM, which includes a slave interface connected to the DMA of an NPU, a master interface connected to the DRAM having a plurality of slave ports, and a switch that connects the slave interface to the master interface, and the connection device receives a data read request from the DMA of the NPU and transfers the received data read request to the DRAM, receives data or a response corresponding to the received data read request from the plurality of slave ports, and rearranges the data or the response based on an order of the received data read request so as to be transferred to the DMA of the NPU.