Propagation latency reduction
By switching matrix operation allocation directions in machine learning accelerators, latency between tiles is reduced, enhancing performance through efficient scheduling techniques.
HK40135014APending Publication Date: 2026-07-17GOOGLE LLC
Patent Information
- Authority / Receiving Office
- HK · HK
- Patent Type
- Applications
- Current Assignee / Owner
- GOOGLE LLC
- Filing Date
- 2026-05-14
- Publication Date
- 2026-07-17
AI Technical Summary
Technical Problem
Machine learning accelerators face inherent latency issues due to computational and propagation delays between tiles, which are not effectively addressed by existing technologies, especially as device size increases.
Method used
A scheduling technique that modifies the allocation direction of matrix operations from row-major to column-major at specific periods to reduce latency, allowing concurrent processing across tiles and hiding propagation delays.
Benefits of technology
This approach reduces computational and propagation latency, achieving performance improvements without requiring hardware changes, with potential near 100% utilization even in single-tile scenarios.
✦ Generated by Eureka AI based on patent content.
Smart Images
Figure 00000000_0000_ABST
Abstract
Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for scheduling operations to reduce propagation delays between tiles of an accelerator. One of the methods includes receiving a request to generate a schedule for a first layer of a program to be executed by an accelerator configured to perform matrix operations at least partially in parallel, where the program defines a plurality of layers including the first layer, each layer of the program defining a matrix operation to be performed using a respective matrix of values. A plurality of scheduled initial blocks are allocated according to the initial allocation direction. The distribution direction is switched starting with a particular period such that the blocks processed after the selected particular period are processed along a different second dimension of the first matrix. And then distributing all the remaining undistributed blocks according to the switched distribution direction.
Need to check novelty before this filing date? Find Prior Art