Multi-Level Multi-Striding DMA Architecture for Parallel Address Generation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing DMA architectures struggle to efficiently perform multi-level multi-striding and determine multiple memory addresses in parallel during a single clock cycle, limiting the offloading of memory transfer operations from processor cores and impacting computational efficiency.
Innovation Solution
The implementation of DMA architectures that support multi-level multi-striding and parallel determination of multiple memory addresses, utilizing DMA threads with separate subthreads for read and write operations, and a request generator capable of generating memory addresses in parallel, allowing for up to four tensor dimensions and addresses per clock cycle, with parameterizable configurations for different dimensions and addresses.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional DMA architectures are used, then device simplicity is maintained, but memory operation throughput is limited
Solution Approach 1:
The DMA architecture is segmented into multiple independent DMA threads (e.g., DMA thread 0, DMA thread 1, etc.), where each thread can independently generate and execute DMA requests. This segmentation enables parallel processing of multiple memory operations simultaneously, doubling the throughput while maintaining manageable complexity through modular design.
Solution Approach 2:
The patent introduces multi-level multi-striding capability that operates across multiple dimensional levels (e.g., first level striding, second level striding). This dimensional expansion allows the DMA architecture to address complex memory access patterns in parallel, increasing throughput without linearly increasing overall system complexity.
2Productivity
If parallel address generation is implemented, then computational efficiency is improved, but device complexity increases
Solution Approach 1:
The address generation function is segmented and distributed across multiple DMA threads, where each thread possesses its own address generation capabilities. This allows parallel determination of multiple memory addresses simultaneously while keeping each individual address generator relatively simple and manageable.
Solution Approach 2:
The DMA threads pre-generate multiple memory addresses in parallel during a single clock cycle before actual memory operations are executed. This preliminary parallel address generation eliminates sequential bottlenecks and improves computational efficiency, with the complexity managed through hardware optimization of the address generation logic.
Data Source
Figure 1
Figure 2A
Figure 2B
AI summary
DMA architectures capable of performing multi-level multi-striding and determining multiple memory addresses in parallel are described. In one aspect, a DMA system includes one or more hardware DMA threads. Each DMA thread includes a request generator configured to generate, during each parallel memory address computation cycle, m memory addresses for a multi-dimensional tensor in parallel and, for each address, a respective request for a memory system to perform a memory operation. The request generator includes m memory address units that each include a step tracker configured to generate, for each dimension of the tensor, a respective step index value for the dimension and, based on the respective step index value, a respective stride offset value for the dimension. Each memory address unit includes a memory address computation element configured to generate a memory address for a tensor element and transmit the request to perform the memory operation.