Sharding Synchronous Processors for Tile Load Balancing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing compiler techniques struggle to optimize the performance of synchronous integrated circuit accelerators, particularly in systems with multiple tiles arranged in a linear array, due to limitations in scheduling operations and managing resource constraints.
Innovation Solution
The described techniques involve advanced sharding methods such as sublayer sharding, parallelogram time slicing, oversharding, and zigzag sharding to optimize the allocation of matrix operations across multiple synchronous tiles, thereby improving performance and load balancing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional compiler techniques are used for scheduling operations on multiple synchronous tiles, then the implementation is simple, but the performance optimization is insufficient
Solution Approach 1:
The patent applies segmentation by dividing the computational task into fine-grained operations that can be independently scheduled across multiple synchronous tiles. Each tile is assigned specific segments of the computation graph, enabling parallel execution while maintaining simplicity through systematic division rather than complex coordination.
Solution Approach 2:
The compiler performs preliminary scheduling actions by pre-computing and assigning operations to specific tiles before execution. This includes determining the execution order of tiles, allocating data to appropriate tiles, and preparing the computation graph in advance, which optimizes performance without requiring complex runtime decision-making.
2Productivity
If operations are scheduled with fine-grained precision at cycle level, then the performance and load balancing improve, but the scheduling complexity increases
Solution Approach 1:
The patent changes the scheduling parameter from coarse-grained task-level scheduling to fine-grained cycle-level scheduling. This involves breaking down operations into atomic cycles and assigning them to specific tiles with precise timing, which maximizes tile utilization and enables optimal load balancing across the synchronous array.
3Productivity
If sharding techniques are applied to distribute operations across tiles, then load balancing improves, but the data communication requirements increase
Solution Approach 1:
The computation and data are segmented and distributed across multiple tiles in a way that minimizes communication. By dividing the computation graph into segments that can be executed in parallel on different tiles, the system achieves load balancing while reducing the need for extensive data movement between tiles.
Solution Approach 2:
The patent introduces an intermediary scheduling layer that coordinates data movement between tiles. This intermediary mechanism manages the communication overhead by intelligently routing data only when necessary, rather than requiring all tiles to communicate with each other, thus reducing overall communication complexity.
4Productivity
If multiple sharding methods are used to optimize different aspects of performance, then the overall performance improves, but the compiler complexity increases
Solution Approach 1:
The patent implements a universal scheduling framework that can handle multiple sharding methods and optimization techniques within a single compiler pass. This multi-functional approach allows the same scheduling infrastructure to support various sharding strategies (e.g., row-major, column-major, diagonal) without requiring separate complex implementations for each method.
Data Source
AI summary
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for sharding dataflow graphs for a device having multiple synchronous tiles. One of the methods includes receiving a representation of a dataflow graph comprising a plurality of nodes that each represent respective matrix operations to be performed by a device having a plurality synchronous tiles. Candidate allocations of respective portions of the dataflow graph to each tile of the plurality of synchronous tiles are evaluated according to one or more resource constraints of the device. One of the candidate allocations is selected based on evaluating each candidate allocation.