Solver-Based Buffer Depth Balancing in Reconfigurable AI Accelerators
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Optimizing compute graphs for the configurable units of coarse-grained reconfigurable processors remains a challenge, particularly in managing dataflow graphs with nested loops and varying execution times, which are crucial for machine learning and artificial intelligence workloads.
Innovation Solution
A method for improving runtime performance by balancing stage buffer depths in a reconfigurable computing system, involving a grid of compute units and memory units, using a global solver-based approach to minimize total associated PMU cost and optimize dataflow paths.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If stage buffer depths are unbalanced in dataflow paths, then place and route issues occur and runtime performance degrades, but adding more buffers increases device complexity and resource usage
Solution Approach 1:
The patent applies parameter changes by systematically adjusting buffer depth parameters across different stages of the dataflow graph. The solver modifies buffer depth values to achieve balance, transforming the system from an unbalanced state (causing performance degradation) to a balanced state (optimizing runtime performance) while managing the complexity of buffer configuration through automated optimization.
2Productivity
If buffer depths are increased to prevent congestion, then dataflow efficiency improves, but memory resource consumption increases
Solution Approach 1:
The patent uses parameter changes to optimize buffer depth values, finding the optimal balance point where dataflow efficiency is maximized without excessive memory consumption. The solver adjusts buffer depth parameters to achieve sufficient dataflow efficiency while preventing both congestion and over-provisioning of memory resources.
Solution Approach 2:
The patent implements feedback through the solver-based optimization process, which evaluates the impact of buffer depth configurations on dataflow efficiency and memory usage. The solver uses this feedback to iteratively adjust buffer depths, converging on an optimal configuration that balances dataflow efficiency with memory resource consumption.
3Device complexity
If buffer depths are decreased to reduce resource usage, then device complexity reduces, but latency increases and throughput decreases
Solution Approach 1:
The patent applies parameter changes by optimizing buffer depth values to find the minimum sufficient depth that maintains acceptable latency and throughput while reducing unnecessary buffer resources. The solver adjusts buffer depth parameters downward from excessive values to optimal values, reducing device complexity without sacrificing performance.
Data Source
AI summary
A method for reducing latency and increasing throughput in a reconfigurable computing system includes receiving a user program for execution on a reconfigurable dataflow computing system, comprising a grid of interconnected compute units and grid of memory units. The user program includes multiple tensor-based algebraic expressions that are converted to an intermediate representation comprising multiple stages. Each stage includes one or more logical operations executable via dataflow through compute units, and each stage is preceded by and followed by a stage buffer, each stage buffer corresponding to one or more memory units. The method includes detecting a final joining stage that consumes a first and second dataflow path, having a first and second total stage buffer depth, respectively, and balancing the first and second total stage buffer depths via tuning or inserting a whole stage buffer, wherein a total associated PMU cost is minimized.