Merging Skip-Buffers for CGR Memory Efficiency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Optimizing compute graphs for the configurable units of coarse-grained reconfigurable (CGR) processors remains a challenge, particularly in efficiently executing massively parallel computations required for machine learning and artificial intelligence workloads.
Innovation Solution
A method is introduced in a reconfigurable computing system where tensor consumers are connected to their corresponding tensor producers via skip-buffers, allowing for the determination and merging of compatible skip-buffers to reduce memory unit consumption and latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If multiple skip-buffers are used to connect tensor consumers to tensor producers, then data flow parallelism and execution flexibility are improved, but memory unit consumption increases
Solution Approach 1:
The patent merges multiple skip-buffers into a single shared skip-buffer when they are compatible, meaning they serve the same tensor producer and their consumer access patterns do not conflict. This consolidation reduces the total number of buffer instances from N to 1, directly decreasing memory unit consumption while preserving the ability to service multiple tensor consumers through the shared buffer.
Solution Approach 2:
The shared skip-buffer is designed to serve multiple tensor consumers simultaneously through broadcast compatibility mechanisms. A single buffer instance can be read by multiple consumers in parallel, allowing the same physical memory resource to fulfill multiple data flow requirements, thus improving resource utilization efficiency.
2Ease of operation
If multiple separate skip-buffers are allocated for different tensor consumers, then data access independence is improved, but device complexity increases
Solution Approach 1:
Compatible skip-buffers are merged into a single shared buffer instance, reducing the total number of buffer management operations. Instead of independently allocating, initializing, and managing N separate buffers, the system manages one shared buffer, significantly simplifying device complexity while maintaining functional independence through controlled access mechanisms.
Solution Approach 2:
The patent introduces compatibility checking mechanisms and access control logic as intermediaries between the shared skip-buffer and multiple tensor consumers. These intermediaries manage data access coordination, ensuring that consumers can independently access the shared buffer without conflicts, thus preserving data access independence while reducing overall system complexity.
3Productivity
If larger buffer depth is used in skip-buffers, then data availability for parallel execution is improved, but memory unit consumption increases
Solution Approach 1:
The patent merges multiple skip-buffers with individual buffer depths into a single shared skip-buffer with an optimized buffer depth. By consolidating buffers, the system achieves the necessary data availability for parallel execution through a single shared resource with sufficient depth, rather than requiring multiple separate buffers each with adequate depth, thus reducing total memory consumption while maintaining parallel execution capability.
Data Source
AI summary
A method in a reconfigurable computing system includes connecting a plurality of tensor consumers to their corresponding tensor producers via skip-buffers, which generates a plurality of skip-buffers. The method includes determining that at least one skip-buffer of the plurality of skip-buffers corresponding to a first set of tensor consumers and at least one skip-buffer of the plurality of skip-buffers corresponding to a second set of tensor consumers, are compatible to wholly or partially merge. The method also includes merging, wholly or partially, the compatible skip-buffers to produce a merged skip-buffer having a minimal buffer depth. The described method may reduce memory unit consumption and latency.