Buffer Splitting for Memory Optimization in Reconfigurable Dataflow Systems
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Optimizing compute graphs for the configurable units of a coarse-grained reconfigurable (CGR) processor remains a challenge, particularly in efficiently executing massively parallel computations required for machine learning and artificial intelligence workloads.
Innovation Solution
A method in a reconfigurable computing system that involves receiving a user program, converting tensor-based algebraic expressions to an intermediate representation, and determining whether splitting a selected buffer yields a reduced cost, with the option to split the buffer and optimize resource utilization and memory consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If buffer splitting is applied to reduce memory unit consumption, then resource utilization is improved, but device complexity increases
Solution Approach 1:
The patent applies segmentation by dividing a single large buffer into multiple smaller buffers. This allows the system to reduce overall memory unit consumption by selectively allocating smaller buffer sizes based on actual computational needs, thereby resolving the contradiction between reducing resource consumption and managing system complexity through structured division.
Solution Approach 2:
The patent utilizes parameter changes by dynamically adjusting buffer sizes and splitting strategies based on computational workload characteristics. By changing buffer parameters (size, location, allocation) adaptively, the system optimizes memory unit consumption while managing complexity through automated parameter adjustment rather than manual configuration.
2Productivity
If buffer splitting is implemented to optimize resource utilization, then productivity is improved, but ease of operation deteriorates
Solution Approach 1:
The patent implements self-service by enabling the buffer splitting mechanism to automatically optimize resource allocation based on computational graphs and workload characteristics. The system performs self-adjustment of buffer configurations without requiring user intervention, thereby improving productivity while maintaining ease of operation through automation.
Solution Approach 2:
The patent applies preliminary action by pre-configuring buffer splitting strategies and resource allocation plans before execution. The system analyzes computational graphs in advance and pre-determines optimal buffer splits, eliminating the need for complex runtime configuration and maintaining operational simplicity while enhancing productivity.
3Loss of substance
If buffer splitting is used to reduce memory consumption, then loss of substance is reduced, but measurement precision requirements increase
Solution Approach 1:
The patent implements feedback mechanisms that continuously monitor memory consumption patterns and computational workload characteristics. This feedback information is used to refine buffer splitting decisions and cost calculations, improving measurement precision while reducing memory resource waste through adaptive optimization based on actual system behavior.
Data Source
AI summary
A method includes receiving a user program for execution on a reconfigurable dataflow computing system comprising a plurality of compute units and a plurality of memory units, the plurality of compute units and the plurality of memory units being interconnected. The user program is converted to an intermediate representation comprising a plurality of logical operations, executable via dataflow through one or more compute units of the plurality of compute units, one or more logical operations preceded by or followed by a buffer of one or more buffers, each buffer of the one or more buffers corresponding to one or more memory units of the plurality of memory units. The method further includes determining whether splitting a selected buffer yields a reduced cost and splitting the selected buffer in response to determining that splitting the selected buffer yields the reduced cost, to produce a first buffer and a second buffer.