Critical-Stage Memory Mapping in Reconfigurable Dataflow Architectures
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Optimizing compute graphs for the configurable units of coarse-grained reconfigurable architectures (CGRAs) remains a challenge, particularly in efficiently executing massively parallel computations required for machine learning and artificial intelligence tasks.
Innovation Solution
A method for optimizing reconfigurable computing systems involves converting user programs with tensor-based algebraic expressions into intermediate representations, analyzing critical stages for memory mapping operations, and moving these operations to adjacent stages executable by memory units, utilizing a grid of compute and memory units connected by a switching array.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If memory mapping operations are performed within the critical stage, then computational accuracy is maintained, but latency increases and throughput decreases
Solution Approach 1:
The patent segments the computational pipeline into distinct stages, moving memory mapping operations out of the critical computational stage and into adjacent stages. This segmentation allows the critical stage to focus solely on computation while memory mapping occurs in parallel or sequentially in non-critical stages, thereby reducing latency without compromising computational accuracy.
Solution Approach 2:
The patent performs memory mapping operations as preliminary actions in stages preceding the critical computational stage. By preparing data layouts and mappings before they are needed for computation, the system eliminates waiting time during the critical path, thus reducing overall latency while maintaining the required data transformations for accurate computation.
2Manufacturing precision
If memory mapping operations are performed within the critical stage, then data transformation accuracy is ensured, but throughput decreases
Solution Approach 1:
The patent divides the processing pipeline into multiple stages, separating memory mapping operations from the critical computational stage. This segmentation enables memory mapping to be performed in parallel or in non-critical paths, allowing the critical stage to maintain high throughput for computations while data transformation accuracy is ensured through dedicated mapping stages that prepare data appropriately.
Solution Approach 2:
The patent introduces intermediate buffer stages that act as mediators between memory operations and computational stages. These intermediary buffers handle memory mapping operations, allowing the critical computational stage to receive pre-processed data without performing mapping operations itself, thus maintaining both throughput and transformation accuracy.
3Device complexity
If the computing system uses a fixed architecture, then hardware simplicity is maintained, but adaptability for different computational patterns decreases
Solution Approach 1:
The patent implements dynamic reconfigurability in the computing system, allowing the architecture to adapt its configuration based on the computational patterns being executed. This dynamic approach enables the system to optimize its structure for different workloads (e.g., changing dataflow patterns, reconfiguring interconnects) while maintaining a relatively simple base hardware design that can be flexibly programmed for different functions.
Data Source
AI summary
A method for reducing latency and increasing throughput in a reconfigurable computing system includes receiving a user program for execution on a reconfigurable dataflow computing system, comprising a grid of compute units and grid of memory units interconnected with a switching array. The user program includes multiple tensor-based algebraic expressions that are converted to an intermediate representation comprising multiple stages. Each stage includes one or more logical operations executable via dataflow through compute units, and each stage is preceded by and followed by a buffer, each buffer corresponding to one or more memory units. The method includes detecting a memory mapping operation within a critical stage and moving the memory mapping operation to an adjacent stage, wherein the memory mapping operation is executable by memory units within the adjacent stage and dataflow through the buffer is controlled by one or more memory units within the grid of memory units.