DMA Transfer Synchronization Using Reference Counting Semaphores
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In neural network processors, the use of a large number of semaphores for synchronization of DMA transfers is costly and complex, leading to performance issues due to the need for numerous hardware registers and concurrent wait instructions, especially when dealing with a large number of DMA queues.
Innovation Solution
Implementing a method that uses a smaller number of semaphores, such as 3 semaphores for synchronization of 16 DMA transfers, by assigning DMA transfers in a round-robin manner and allowing waiting engines to wait on a single semaphore for multiple DMA transfers, reducing the number of concurrent wait instructions and hardware registers required.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a large number of semaphores are used for synchronization of DMA transfers, then synchronization coverage is improved, but device complexity and chip area increase
Solution Approach 1:
The patent merges multiple semaphore operations into a single semaphore by implementing a reference counting mechanism. Instead of using one semaphore per DMA queue, the system uses a single semaphore with an associated counter that tracks the number of active DMA transfers. When a DMA transfer completes, the counter is decremented and the semaphore is signaled only when the counter reaches zero, indicating all transfers are complete. This combining approach maintains synchronization coverage while reducing the number of semaphore hardware registers from N (number of queues) to 1.
Solution Approach 2:
The single semaphore structure is designed to serve multiple DMA queues simultaneously through the reference counting mechanism. The semaphore becomes a universal synchronization primitive that can coordinate any number of DMA transfers across multiple queues by maintaining a dynamic count of active transfers. This multi-functional design allows the same semaphore hardware to replace multiple dedicated semaphores, reducing chip area while maintaining the ability to synchronize numerous concurrent DMA operations.
2Reliability
If a large number of semaphores are used for synchronization, then all DMA queues can be synchronized, but hardware register quantity and cost increase
Solution Approach 1:
The patent combines N separate semaphore hardware registers into a single semaphore register by implementing a reference counting system. The single semaphore register maintains a counter value that represents the number of incomplete DMA transfers across all queues. This merging reduces the hardware register quantity from N registers to 1 register, directly addressing the cost and area concerns while preserving the synchronization capability for all DMA queues through the unified counter mechanism.
3Reliability
If numerous concurrent wait instructions are used for semaphore synchronization, then all DMA transfers are coordinated, but execution efficiency decreases
Solution Approach 1:
The patent merges multiple concurrent wait instructions into a single wait instruction by using a unified reference counting semaphore. Instead of having each DMA transfer increment and signal its own separate semaphore (requiring N concurrent wait operations), the system uses a single counter that aggregates the state of all DMA transfers. Engines issue a single wait instruction on this unified semaphore, and the counter automatically tracks when all transfers are complete, reducing the instruction count from N to 1 and improving execution efficiency.
Solution Approach 2:
The reference counting mechanism operates autonomously to track DMA transfer completion without requiring explicit wait instructions for each transfer. The counter is automatically incremented when DMA transfers start and decremented when they complete, allowing the system to self-manage the synchronization state. This self-service approach reduces the burden on executing engines to manually manage multiple wait instructions, as the unified counter automatically reflects the overall synchronization state.
Data Source
AI summary
Synchronization of a plurality of aggregate DMA transfers on large number of DMA queues can be achieved using a small number of semaphores. One or more semaphores from M semaphores can be assigned to each aggregate DMA transfer based on round-robin or another suitable method. Each aggregate DMA transfer can comprise N DMA transfers, where M is smaller than N. Each DMA transfer can be assigned to one of the assigned one or more semaphores from the M semaphores. Each DMA engine of N DMA engines can increment the assigned semaphore after performing a respective DMA transfer of the N DMA transfers. A computational engine waiting on completion of a certain aggregate DMA transfer can perform an operation based upon the one or more assigned semaphores reaching respective threshold values.