MLA Sync Detector Clock Gating for Static Schedule
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Machine learning networks face challenges in being implemented on edge devices due to resource limitations such as memory and processing capacity, which complicates the synchronization of compute elements executing statically scheduled instructions, especially when dealing with off-chip memory with variable access times.
Innovation Solution
A Machine Learning Accelerator (MLA) system that includes a sync detector and sync controller to suspend execution if sufficient instructions and data are not transferred from off-chip memory, ensuring synchronization by temporarily stopping the clock through clock gating, allowing for deterministic data and instruction transfers within the MLA's mesh.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Use of energy by moving object
If compute elements are simplified to conserve resources on edge devices, then resource consumption is reduced, but synchronization complexity increases
Solution Approach 1:
Each compute element includes an integrated sync detector that autonomously monitors instruction availability and generates sync requests when needed, eliminating the need for complex centralized synchronization control logic
Solution Approach 2:
The synchronization function is divided into independent components distributed across each compute element (sync detector, instruction queue, clock gate), allowing each element to operate independently while maintaining overall system synchronization
2Productivity
If statically scheduled instructions are used in MLA, then execution efficiency is improved, but handling variable access times from off-chip memory becomes difficult
Solution Approach 1:
Instructions are pre-loaded into each compute element's instruction queue before execution begins, and the sync detector continuously monitors queue depth to ensure sufficient instructions are available before each scheduled execution phase
Solution Approach 2:
The sync detector provides continuous feedback on instruction queue status and generates sync requests to the sync controller when instructions are not available, allowing the system to adapt to variable memory access times while maintaining static scheduling
3Reliability
If sync detector is integrated in each compute element, then synchronization monitoring is improved, but device complexity increases
Solution Approach 1:
The sync detector leverages existing compute element components (instruction queue, clock gate) to perform synchronization monitoring, making the same structural elements serve both execution control and synchronization functions
Solution Approach 2:
The sync detector is integrated within the compute element structure, combining synchronization monitoring functionality with the existing instruction execution pipeline and clock control mechanisms
Data Source
AI summary
A machine learning accelerator (MLA) implemented on a semiconductor die includes a computing mesh of interconnected compute elements that includes storage elements (SEs) and processing elements (PEs). The compute elements execute a program of instructions to implement a machine learning network according to a static schedule for execution of the instructions. The instructions include data transfer instructions and compute instructions. The MLA includes a memory interface to off-chip memory. The MLA fetches instructions for the PEs from the off-chip memory, and the MLA transfers data between the SEs and the off-chip memory. A sync detector determines, for each compute element, whether sufficient data and instructions are available for continued operation of the compute element according to the static schedule. It generates a sync request if sufficient data and/or instructions are not available. A sync controller suspends operation of the computing mesh in response to the sync request.