Neural Processor Synchronization with Distributed L3 Sync Targets
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional neural processing units suffer from high latency and increased overhead due to centralized control processors managing synchronization signals, which becomes exacerbated with the inclusion of more processing units and cores.
Innovation Solution
A neural processing device with multiple neural processors that perform synchronization independently through L3 sync targets, shared memory, semaphore memories, and a global interconnection, eliminating the need for a centralized control processor and reducing scheduling overhead.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a centralized control processor manages synchronization signals, then coordination between processing units is achieved, but latency increases and control processor overhead increases
Solution Approach 1:
The patent divides the centralized control function into distributed control elements. Each processing unit is equipped with its own control logic and synchronization capabilities, allowing autonomous decision-making without centralized coordination. This segmentation eliminates the single point of control that causes latency and overhead.
Solution Approach 2:
The patent merges the control processor functions directly into each processing unit. By combining control and computation capabilities within each unit, the system eliminates the need for separate centralized control signaling, thereby reducing latency and overhead while maintaining reliable synchronization.
2Productivity
If more processing units and cores are included, then computing power increases, but synchronization overhead increases
Solution Approach 1:
Each processing unit is designed to be self-sufficient with embedded control logic that enables autonomous synchronization decisions. This self-service capability allows processing units to coordinate with each other directly without requiring centralized management, thereby maintaining low overhead even as the number of units increases.
Solution Approach 2:
The control functionality is segmented and distributed across all processing units rather than centralized. This distribution allows each unit to handle its own synchronization needs independently, preventing the accumulation of overhead that would occur with centralized control of numerous units.
3Reliability
If a centralized control processor manages synchronization, then coordination is achieved, but control processor overhead increases
Solution Approach 1:
The control function is segmented and distributed to each processing unit, eliminating the need for a power-consuming centralized control processor. Each unit's local control logic consumes minimal energy compared to a centralized processor managing all synchronization for multiple units.
Solution Approach 2:
Processing units perform their own synchronization control locally without requiring centralized management. This self-service approach eliminates the overhead energy consumption of a dedicated control processor while maintaining reliable coordination between units.
Data Source
AI summary
A neural processing device is provided. The neural processing device comprises a plurality of neural processors, a shared memory shared by the plurality of neural processors, a plurality of semaphore memories, and global interconnection. The plurality of neural processors generates a plurality of L3 sync targets, respectively. Each semaphore memory is associated with a respective one of the plurality of neural processors, and the plurality of semaphore memories receive and store the plurality of L3 sync targets, respectively. Synchronization of the plurality of neural processors is performed according to the plurality of L3 sync targets. The global interconnection connects the plurality of neural processors with the shared memory, and comprises an L3 sync channel through which an L3 synchronization signal corresponding to at least one L3 sync target is transmitted.