Neural Core Synchronization Using Programmable Transactional Memory
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional neural processing units face synchronization latency and increased control processor overhead due to centralized synchronization management, which becomes inefficient as the number of processing units and cores increases.
Innovation Solution
A neural processing device with a programmable hardware transactional memory (PHTM) that manages synchronization by directly processing memory access requests from multiple neural cores, using a shared memory and L2 sync path for efficient synchronization signal transmission.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If a centralized control processor manages synchronization for neural cores, then synchronization control is consolidated, but synchronization latency and control processor overhead increase
Solution Approach 1:
The patent segments the centralized control processor into distributed memory controllers, each independently managing synchronization for specific neural core groups. This segmentation reduces the control burden on any single processor while maintaining coordinated synchronization across all cores through shared memory transactions.
Solution Approach 2:
The patent introduces hardware transactional memory (HTM) as an intermediary mechanism between neural cores and shared memory. The HTM handles memory access requests and synchronization coordination automatically, reducing the overhead on control processors while maintaining proper synchronization timing.
2Productivity
If more processing units and cores are included in the neural processing unit, then computing capability increases, but synchronization processing latency and control processor overhead increase
Solution Approach 1:
The patent divides the neural processing unit into multiple independent neural core groups, each with its own memory controller. This segmentation allows parallel synchronization management across different core groups, reducing overall synchronization latency while supporting increased computing capability through more cores.
Solution Approach 2:
The patent enables memory controllers to autonomously manage synchronization for their respective neural core groups without requiring constant intervention from a centralized control processor. Each memory controller independently handles memory access requests and synchronization signals for its assigned cores.
3Ease of operation
If centralized control processor manages all synchronization signals, then coordination is simplified, but hardware complexity and overhead increase
Solution Approach 1:
The patent segments the synchronization management function across multiple distributed memory controllers rather than concentrating it in a single centralized processor. Each memory controller manages synchronization for its local neural core group, simplifying the operational complexity while distributing the hardware complexity across multiple simpler units.
Solution Approach 2:
The patent uses hardware transactional memory as an intermediary that automatically handles synchronization coordination between neural cores and shared memory. This intermediary mechanism simplifies the operational complexity by providing transparent memory access while managing the hardware complexity through dedicated synchronization hardware paths.
Data Source
AI summary
A neural processing device and transaction tracking method thereof are provided. The neural processing device comprises a first set of a plurality of neural cores, a shared memory shared by the first set of the plurality of neural cores, and a programmable hardware transactional memory (PHTM) configured to receive a memory access request directed to the shared memory from the first set of the plurality of neural cores and configured to commit or buffer the memory access request.