Bandwidth-Aware CGR Graph Mapping for Lower Data Transfer Overhead
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing architectures such as CPUs and GPUs face challenges in efficiently executing complex algorithms like deep learning models due to limitations in data flow management, leading to excessive data transfers and poor hardware utilization, especially when scaling to large compute clusters.
Innovation Solution
A method for transforming high-level programs to map onto coarse-grained reconfigurable (CGR) processors by sectioning dataflow graphs, assigning computations based on memory bandwidth, eliminating buffers, and generating configuration data for CGR units to implement the dataflow graph, optimizing communication channels.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If dataflow graphs are executed on traditional CPU/GPU architectures, then general-purpose computing is achieved, but data transfer overhead increases and hardware utilization decreases
Solution Approach 1:
The patent segments the dataflow graph into multiple sections and maps different sections to different CGR units. This segmentation allows computations to be distributed across multiple reconfigurable units, reducing centralized data transfer bottlenecks and improving parallel execution efficiency while minimizing energy consumption from excessive data movement between memory and processing units.
Solution Approach 2:
The patent introduces on-chip memory and buffer structures as intermediaries between the CGR units and external memory. These intermediary structures reduce the frequency and volume of data transfers to external memory, thereby reducing data transfer overhead and energy consumption while maintaining efficient computation flow across the segmented graph sections.
2Reliability
If buffers are added to manage data flow between computations, then data dependency management is improved, but hardware area and complexity increase
Solution Approach 1:
The patent applies local quality by making buffer structures reconfigurable and adaptive rather than static. Different buffer configurations are applied locally to different sections of the dataflow graph based on their specific data dependency requirements. This allows minimal buffering where needed while avoiding unnecessary buffers in other sections, reducing overall hardware complexity while maintaining reliable data dependency management.
Solution Approach 2:
The patent makes the buffer structures dynamic and reconfigurable, allowing them to adapt their size and configuration based on the specific computational requirements of each graph section. This dynamic approach enables the system to optimize buffer usage for each local context, managing data dependencies effectively without committing to a fixed complex buffer architecture throughout the entire system.
3Speed
If course-grained blocks are used in CGRA, then area and delay are reduced for specific functions, but area is wasted when blocks are not used
Solution Approach 1:
The patent segments the computational graph and maps different sections to different CGR units, enabling fine-grained utilization of course-grained blocks. By dividing the workload into manageable sections that can be distributed across multiple units, the system maximizes the utilization of each course-grained block, reducing wasted area while maintaining the speed advantages of coarse-grained architecture for active computations.
Solution Approach 2:
The patent enhances the universality of CGR units by making them reconfigurable to perform different functions based on the specific section of the dataflow graph being executed. This multi-functionality allows course-grained blocks to be dynamically allocated to different computational tasks, reducing area waste when blocks are not used for their original intended function while maintaining high execution speed for the current task.
Data Source
AI summary
A computer-implemented method of transforming a high-level program for mapping onto a coarse-grained reconfigurable (CGR) processor with an array of CGR units, including sectioning a dataflow graph into a plurality of sections; extracting performance information for each of the plurality of sections; on a CGR unit: assigning to a section at least two computations dependent on a first data element; scheduling an additional load of the first data element in response to available memory bandwidth for that section; eliminating a buffer between the additional load of the first data element and one of the two computations, for that section; generating configuration data for the and communication channels, wherein the configuration data, when loaded onto an instance of the array of CGR units, causes the array of CGR units to implement the dataflow graph; and storing the configuration data in a non-transitory computer-readable storage medium.