Heterogeneous Computing Data Flow Using Partitioned Memory Buffers
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing heterogeneous computing systems inefficiently manage data flow between different types of processing devices, leading to unnecessary data copying and rigid data synchronization mechanisms.
Innovation Solution
Implementing data flow management by determining dependencies between operations across heterogeneous processing devices, initializing buffer partitions, and optimizing memory management operations to reduce unnecessary data synchronization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If existing heterogeneous computing runtime manages data flow between different types of processing devices, then data transfer between stages can be achieved, but unnecessary data is copied back and forth between memory devices and synchronization overhead increases
Solution Approach 1:
The patent segments memory access by introducing buffer partitions that are specifically allocated for cross-device data transfer. Instead of treating all memory operations uniformly, the system divides memory into distinct regions (buffer partitions) that facilitate direct access between heterogeneous processing devices, thereby reducing unnecessary synchronization overhead while maintaining efficient data transfer.
2Reliability
If existing data synchronization mechanism always involves host memory, then data can be synchronized between processing devices, but the mechanism becomes rigid and inefficient
Solution Approach 1:
The patent introduces buffer partitions as intermediary structures that enable direct data access between heterogeneous processing devices without requiring host memory involvement. These buffer partitions act as mediators that facilitate efficient data transfer while maintaining synchronization reliability, thereby reducing the rigidity of the existing mechanism that always routes through host memory.
3Productivity
If buffer partitions are initialized for every cross-boundary dependency, then direct access between heterogeneous processing devices can be enabled, but memory management operations increase
Solution Approach 1:
The patent applies preliminary action by initializing buffer partitions in advance for cross-boundary dependencies before data transfer operations begin. This pre-initialization allows heterogeneous processing devices to directly access allocated buffer regions without requiring runtime memory management operations, thereby enabling efficient direct access while actually reducing the overall number of memory management operations needed during execution.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Embodiments include computing devices, apparatus, and methods implemented by the apparatus for implementing data flow management on a computing device. Embodiment methods may include initializing a buffer partition of a first memory of a first heterogeneous processing device for an output of execution of a first iteration of a first operation by the first heterogeneous processing device on which a first iteration of a second operation assigned for execution by a second heterogeneous processing device depends. Embodiment methods may include identifying a memory management operation for transmitting the output by the first heterogeneous processing device from the buffer partition as an input to the second heterogeneous processing device. Embodiment methods may include allocating a second memory for storing data for an iteration executed by a third heterogeneous processing device to minimize a number of memory management operations for the second allocated memory.