Compute-Graph Loop Fusion for Higher Data Processor Throughput
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Optimizing compute graphs for coarse-grained reconfigurable processors remains a challenge, particularly in efficiently executing massively parallel computations required for machine learning and artificial intelligence tasks.
Innovation Solution
A method is introduced to detect and fuse common operations within nested meta-pipeline loops in a reconfigurable dataflow processor, enhancing throughput by converting algebraic representations into a single operation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If common operations in nested meta-pipeline loops are executed separately, then the implementation is simpler, but compute throughput is reduced
Solution Approach 1:
The patent merges common operations from inner and outer meta-pipeline loops into a single fused operation. Specifically, when the same operation appears in both nested loops, the compiler identifies these as candidates for fusion and combines them to execute once with updated parameters, thereby improving compute throughput by eliminating redundant executions while managing the complexity through systematic detection and fusion rules
Solution Approach 2:
The patent performs preliminary detection and analysis of common operations between nested meta-pipeline loops before execution. The compiler statically identifies operations that appear in both inner and outer loops, determines their fusion eligibility, and prepares the fused operation structure in advance, which enables optimized throughput without runtime overhead while controlling implementation complexity through compile-time analysis
2Productivity
If nested meta-pipeline loops are optimized for parallel execution, then throughput improves, but operation redundancy increases
Solution Approach 1:
The patent combines redundant operation instances from nested loops by detecting common operations that appear in both inner and outer meta-pipeline loops. When an operation is identified as common, it is fused into a single execution with aggregated parameters, reducing the total quantity of operation instances while maintaining parallel execution throughput through efficient resource utilization
3Productivity
If operation fusion is implemented in nested loops, then compute efficiency improves, but detection complexity increases
Solution Approach 1:
The patent performs preliminary detection of common operations during the compilation phase before code generation. The compiler analyzes the algebraic representation of nested meta-pipeline loops, identifies operations that appear in both inner and outer loops, and determines their fusion eligibility in advance. This compile-time detection approach improves compute efficiency through fusion while managing detection complexity by performing the analysis once during compilation rather than during execution
Data Source
AI summary
The present disclosure provides a system for improving high-level program execution throughput. In one aspect, the system detects, within a high-level program executed on a processor, an outer data-processing loop that includes a first data-processing operation. The system further detects, within the high-level program, an inner data-processing loop nested in the outer data-processing loop, wherein the inner data-processing loop includes a second data-processing operation that is interdependent with the first data-processing operation. Next, the system determines if the first data-processing operation and the second data-processing operation performs a common operation. If so, the system fuses the first data-processing operation and the second data-processing operation into a single operation within the inner meta-pipeline loop. As a result, the system increases a throughput in the processor when executing the high-level program by eliminating redundant operations. In some embodiments, the high-level program includes a machine learning (ML) model configured as a dataflow graph.