Neural Core Transaction Tracking With Programmable Hardware Memory
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional neural processing units face significant latency and overhead due to centralized synchronization processing by control processors, which becomes inefficient as the number of processing units and cores increases, leading to delays in task synchronization.
Innovation Solution
The neural processing device employs a programmable hardware transactional memory (PHTM) to manage memory access requests directly, allowing neural cores to operate independently and minimize synchronization waiting time through transaction regions and grouping, reducing hardware complexity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a control processor centrally controls synchronization signals in a neural processing unit, then synchronization management is achieved, but synchronization latency increases and control processor overhead increases as more processing units and cores are included
Solution Approach 1:
The patent divides the centralized control processor into distributed synchronization management units, where each processing unit has its own synchronization controller. This segmentation eliminates the single-point bottleneck, allowing parallel synchronization operations across multiple processing units, thereby reducing synchronization latency while maintaining reliable synchronization management.
Solution Approach 2:
The patent introduces a synchronization buffer as an intermediary component between processing units and the control processor. This buffer temporarily stores synchronization signals, allowing processing units to proceed independently while the control processor manages synchronization at a higher level, thus reducing critical path latency without sacrificing synchronization reliability.
2Reliability
If a control processor centrally controls synchronization signals in a neural processing unit, then synchronization management is achieved, but control processor overhead increases as more processing units and cores are included
Solution Approach 1:
The patent segments the control processor into distributed synchronization controllers embedded within each processing unit cluster. This distributes the control overhead across multiple lightweight units rather than concentrating it in a single processor, reducing the burden on the main control processor while maintaining centralized coordination capability for complex synchronization scenarios.
Solution Approach 2:
The patent implements self-service synchronization mechanisms where processing units autonomously manage their own synchronization state through local buffers and status registers. This self-service approach eliminates the need for the control processor to handle every synchronization event, significantly reducing control overhead while maintaining reliable synchronization through periodic coordination and error checking.
3Device complexity
If memory access requests are managed by software rather than hardware, then hardware complexity is reduced, but synchronization delays occur as tasks must wait for completion depending on dependency
Solution Approach 1:
The patent implements preliminary hardware validation of memory access requests by checking address dependencies and transaction conflicts before execution. This preliminary hardware checking prevents serialization bottlenecks by identifying independent operations that can proceed in parallel, reducing synchronization waiting time while maintaining manageable hardware complexity through targeted validation logic.
Solution Approach 2:
The patent introduces a hardware transactional memory buffer as an intermediary between software memory management and actual memory access. This buffer hardware-component tracks transaction states and coordinates access without requiring complex hardware arbitration logic, thus reducing synchronization delays while keeping hardware complexity acceptable through software-hardware cooperation.
Data Source
Figure 1~2
Figure 3~4
Figure 5~6
AI summary
A neural processing device and transaction tracking method thereof are provided. The neural processing device comprises a first set of a plurality of neural cores, a shared memory shared by the first set of the plurality of neural cores, and a programmable hardware transactional memory (PHTM) configured to receive a memory access request directed to the shared memory from the first set of the plurality of neural cores and configured to commit or buffer the memory access request.