Programmable Hardware Transactional Memory for Neural Core Synchronization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional neural processing units face latency and increased overhead in synchronization processing due to centralized control processors, and software-based management methods can lead to delays in task execution as the number of processing units and cores increases.
Innovation Solution
A neural processing device with a programmable hardware transactional memory (PHTM) that efficiently manages synchronization by receiving memory access requests, committing or buffering them, and using a multi-level synchronization path for parallel processing among neural cores, thereby reducing latency and overhead.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a control processor or centralized controller is used to manage synchronization signals, then synchronization control is achieved, but latency increases and control processor overhead increases
Solution Approach 1:
The patent extracts the synchronization management function from the centralized control processor and implements it in dedicated hardware circuits within each neural core. Each neural core includes a transactional memory module that independently manages its own memory access requests and synchronization, eliminating the need for centralized control and reducing latency.
Solution Approach 2:
The patent introduces a transactional memory module as an intermediary between neural cores and shared memory. This module handles memory access requests locally within each neural core, acting as a mediator that resolves synchronization issues without requiring centralized control, thereby reducing both latency and control processor overhead.
2Productivity
If more processing units and cores are included in the neural processing unit, then computing performance improves, but synchronization overhead increases
Solution Approach 1:
The patent segments the synchronization management function into independent transactional memory modules within each neural core. Each module operates autonomously to manage its own memory access requests, allowing the system to scale to more processing units without proportionally increasing synchronization overhead, as each segment handles its own coordination independently.
3Device complexity
If software-based synchronization management is used, then hardware complexity is reduced, but task execution delays occur
Solution Approach 1:
The patent replaces software-based synchronization management with hardware-based transactional memory modules. These hardware circuits directly manage memory access requests and synchronization at the circuit level, eliminating the delays inherent in software processing while maintaining relatively simple hardware architecture through efficient circuit design.
Data Source
AI summary
A neural processing device and transaction tracking method thereof are provided. The neural processing device comprises a first set of a plurality of neural cores, a shared memory shared by the first set of the plurality of neural cores, and a programmable hardware transactional memory (PHTM) configured to receive a memory access request directed to the shared memory from the first set of the plurality of neural cores and configured to commit or buffer the memory access request.