Neural Network DMA Address Translation for Descriptor Reuse
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Neural network operations, such as inferencing and training, involve a large number of memory access operations between host and local memories, leading to substantial delays due to the creation and management of numerous DMA descriptors, which reduces the overall efficiency of these operations.
Innovation Solution
Implement a neural network hardware accelerator with a DMA engine that uses a data DMA descriptor queue and an address translation table to translate indirect addresses to physical addresses, allowing reuse of DMA descriptors across multiple operations, thereby reducing the need to create new descriptors for each operation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If DMA descriptors are created and managed for each memory access operation, then data transfer between host memory and local memory can be performed, but the creation and management of numerous DMA descriptors causes substantial delays and reduces operational efficiency
Solution Approach 1:
The patent applies preliminary action by pre-configuring DMA descriptors with indirect addresses that can be reused across multiple operations. The address translation table is prepared in advance to map these indirect addresses to physical addresses, eliminating the need to create new descriptors for each operation and reducing descriptor creation delays.
Solution Approach 2:
The patent implements universality by designing DMA descriptors that can serve multiple functions and be reused across different operations. A single descriptor with an indirect address can trigger multiple DMA operations by translating to different physical addresses through the address translation table, allowing one descriptor to perform the work of multiple traditional descriptors.
2Productivity
If traditional DMA operations are used for each neural network operation, then data transfer is achieved, but the large number of operations involves substantial overhead and reduces overall system performance
Solution Approach 1:
The patent uses copying by creating a virtual copy of the address space through indirect addresses. Instead of managing multiple physical address descriptors, the system copies the addressing logic into the descriptor itself via indirect addresses, which are then resolved through the address translation table. This reduces descriptor management complexity while maintaining transfer efficiency.
3Loss of time
If indirect addresses are used in DMA descriptors with address translation, then descriptor reuse is enabled and creation delays are reduced, but an address translation table and additional translation operations are required
Solution Approach 1:
The patent introduces an intermediary - the address translation table - that mediates between indirect addresses in descriptors and physical addresses in memory. This intermediary enables descriptor reuse by translating a single indirect address to multiple possible physical addresses, reducing descriptor creation time while managing the added complexity through a centralized translation mechanism.
Data Source
AI summary
In one example, an apparatus comprises: a direct memory access (DMA) descriptor queue that stores DMA descriptors, each DMA descriptor including an indirect address; an address translation table that stores an address mapping between indirect addresses and physical addresses; and a DMA engine configured to: fetch a DMA descriptor from the DMA descriptor queue to the address translation table to translate a first indirect address of the DMA descriptor to a first physical address based on the address mapping, and perform a DMA operation based on executing the DMA descriptor to transfer data to or from the first physical address.