CGRP Gradient Reduction Using Peer-to-Peer DMA Transfers

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing reconfigurable processor systems face significant communication overhead when implementing data-parallel training of neural networks, as they require external host processors to manage data sharing between reconfigurable processors, leading to inefficiencies in performance and communication bandwidth.

Innovation Solution

A coarse-grained reconfigurable processor system that enables direct peer-to-peer communication between reconfigurable processors using Ethernet Direct Memory Access (EDMA) transactions, allowing for lossless data transfer without external host intervention, thereby optimizing dataflow operations and reducing communication latency.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Ease of operation

If external host processors are used to manage data sharing between reconfigurable processors, then system control and management are simplified, but communication overhead and latency increase

Engineering Contradiction:

Improvesystem controlVSAvoidcommunication latency

Core Design Contradiction:

Ease of operationVSLoss of time

Solution Approach 1:

The reconfigurable processor system performs gradient reduction operations autonomously without requiring external host processor intervention. The system uses internal DMA engines and work queues to manage data transfer and synchronization between reconfigurable processors, enabling self-service operation that eliminates communication overhead with external hosts while maintaining simplified control through automated internal protocols

Inventive Principle:

Principle #25Self-service

2Ease of operation

If external host processors manage data transfer between reconfigurable processors, then data sharing is coordinated, but communication bandwidth is reduced

Engineering Contradiction:

Improvedata sharing coordinationVSAvoidcommunication bandwidth

Core Design Contradiction:

Ease of operationVSQuantity of substance

Solution Approach 1:

The patent introduces DMA (Direct Memory Access) engines as intermediary components that enable direct data transfer between reconfigurable processors without involving external host processors. The DMA engines act as mediators that coordinate data sharing internally within the reconfigurable system, allowing high-bandwidth communication while maintaining organized data transfer through work queue management and completion protocols

Inventive Principle:

Principle #24Intermediary (Mediator)

Data Source

PatentUS12450167B1Autonomous gradient reduction in a reconfigurable processor system

Publication Date: 2025.10.21 SAMBANOVA SYSTEMS INC

AI summary

A coarse-grained reconfigurable processor (CGRP) system for implementing data-parallel training of a neural network is presented. The CGRP system includes a set of coarse-grained reconfigurable units (CGRUs) in a first CGRP configured to implement at least a portion of the neural network, to determine first and second gradients, respectively, of first and second model parameters based on a batch of training data, and to store the first and second gradients in a memory, a network interface including an external direct memory access (DMA) engine coupled between the memory and a network, and a work queue associated with the external DMA engine, wherein completion of determining the first gradient triggers a first work queue entry of the work queue that directs the external DMA engine to transfer the first gradient from the memory over the network to another memory coupled to a second CGRP for a gradient reduction operation.