Input Activation Scheduling in Multi-Tile Machine-Learning Accelerators
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Machine-learning accelerators face inefficiencies due to high data throughput limitations, particularly from repeated memory reads, which affect computational performance and energy efficiency.
Innovation Solution
The implementation of delay registers and scheduling logic in machine-learning accelerators ensures that input values are read only once per row or overall, optimizing data throughput by alternating between activation lines and delay registers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If input values are read from memory for each computation operation, then computational operations can be performed, but data throughput is limited due to high latency and repeated reads
Solution Approach 1:
The patent applies preliminary action by pre-loading input values into delay registers before they are needed for computation. The system reads input values from memory once and stores them in delay registers, which then supply the values for multiple computation operations without requiring additional memory reads. This eliminates the need for repeated memory access and reduces latency.
Solution Approach 2:
The patent introduces delay registers as intermediary storage between memory and the computational units. These registers act as a buffer that holds input values temporarily, allowing the computational units to access them repeatedly without direct memory access. This intermediary layer decouples the memory access timeline from the computation timeline, improving throughput.
2Productivity
If the same input values are reloaded from memory for each use, then computational accuracy is maintained, but hardware utilization efficiency decreases due to redundant reads
Solution Approach 1:
The system performs preliminary action by loading input values into delay registers in advance. Each input value is read from memory once and then reused multiple times from the delay register for different computation operations. This eliminates redundant memory reads and the associated energy consumption while maintaining data accuracy.
3Productivity
If delay registers are used to store input values, then redundant memory reads are eliminated, but device complexity increases due to additional register infrastructure
Solution Approach 1:
The patent applies segmentation by dividing the delay register infrastructure into multiple independent registers, each associated with specific computational units. This modular approach allows the system to scale the register infrastructure according to specific needs without requiring a monolithic complex structure. Each register can be independently controlled and managed.
Solution Approach 2:
The delay registers are designed to serve multiple functions: they store input values for computation, provide timing control for data flow, and enable reuse of data across different computational operations. This multi-functionality reduces the need for separate dedicated structures for each function, thereby managing complexity.
Data Source
Figure 1
Figure 2
Figure 3A
AI summary
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for scheduling operations on a machine-learning accelerator having multiple tiles. The apparatus includes a processor having a plurality of tiles and scheduling circuitry that is configured to select a respective input activation for each tile of the plurality of tiles from either an activation line for the tile or a delay register for the activation line.