Gap Counters for MLA Compute Element Synchronization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Machine learning networks face challenges when implemented on edge devices due to resource limitations such as memory and processing capacity, particularly with the synchronization of compute elements executing statically scheduled instructions, as off-chip memory access times are unpredictable, leading to potential desynchronization and inefficiencies.
Innovation Solution
The implementation of a Machine Learning Accelerator (MLA) with gap counters and a sync controller to maintain synchronization among compute elements by suspending execution when necessary, ensuring that instructions are executed according to a statically scheduled program, even when data or instructions are not immediately available from off-chip memory.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If compute elements execute instructions fetched from off-chip memory, then more data can be processed, but unpredictable access times cause desynchronization between compute elements
Solution Approach 1:
The system performs preliminary actions by pre-fetching instructions and data into on-chip memory before they are needed for execution. The instruction/data manager proactively manages memory access and prepares data in advance, allowing compute elements to execute instructions at predictable intervals without waiting for unpredictable off-chip memory access times.
Solution Approach 2:
On-chip memory serves as an intermediary buffer between off-chip memory and compute elements. It decouples the unpredictable off-chip memory access from the synchronized execution requirements of compute elements, allowing instructions to be fetched in advance and stored locally for reliable, timed access during execution.
2Reliability
If compute elements wait for instructions/data from off-chip memory, then data accuracy is maintained, but execution time increases and power consumption rises
Solution Approach 1:
The instruction/data manager performs preliminary data retrieval from off-chip memory and stores it in on-chip memory before execution is needed. This advance preparation eliminates waiting time during actual execution, as compute elements can immediately access pre-loaded instructions and data from fast on-chip memory without incurring delays.
Solution Approach 2:
The system dynamically adapts by using a hybrid memory approach: on-chip memory for fast, immediate access during execution and off-chip memory for bulk storage. The instruction/data manager dynamically manages data transfer between these memory hierarchies, optimizing the balance between data availability and execution speed based on real-time needs.
3Productivity
If compute elements operate independently without synchronization control, then processing speed increases, but synchronization is lost and execution becomes inefficient
Solution Approach 1:
The sync controller implements a feedback mechanism that monitors the execution state of compute elements and their gap counters. When desynchronization is detected or predicted, the sync controller sends sync commands to pause gap counter incrementing, providing feedback control that maintains synchronization while allowing compute elements to operate at high speed independently.
Solution Approach 2:
The system replaces complex mechanical synchronization mechanisms (such as inter-processor interrupts or message passing) with a simpler software-based gap counter system managed by the sync controller. This substitution reduces overhead and maintains synchronization with minimal interference in the high-speed execution flow of compute elements.
4Reliability
If complex synchronization mechanisms are implemented, then synchronization reliability improves, but device complexity and power consumption increase
Solution Approach 1:
The patent replaces complex hardware synchronization mechanisms with a software-based gap counter system controlled by sync commands. Instead of using intricate inter-processor communication hardware or complex timing circuits, the system uses simple software-managed counters and pause/resume control, significantly reducing device complexity while maintaining synchronization reliability.
Solution Approach 2:
The system changes the synchronization approach from continuous hardware-based timing signals to discrete software-controlled gap counter parameters. By managing synchronization through software parameters (gap counts, sync commands, pause/resume control) rather than fixed hardware timing mechanisms, the system achieves reliable synchronization with reduced complexity and lower power consumption.
Data Source
AI summary
A machine learning accelerator (MLA) implemented on a semiconductor die includes a computing mesh of interconnected compute elements. The compute elements execute a program of instructions to implement a machine learning network according to a static schedule for execution of the instructions. The compute elements include gap counters. The number of cycles between any two instructions (i.e., the gap count) in a statically scheduled program is known and fixed. A gap counter counts cycles during execution and must reach the expected gap count before the later instruction can be executed. Synchronization between different processing elements may be maintained by suspending counting for a period of time.