Subbuffer Objects for Heterogeneous CPU GPU Data Parallelism
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data processing systems struggle to leverage processing resources from both CPUs and GPUs effectively, as they operate in separate programming environments and require vendor-specific interfaces, making it difficult to perform data parallel computations across heterogeneous compute units.
Innovation Solution
A method and apparatus that create subbuffers from a parent buffer for each compute unit, allowing data to be copied and updated across different compute units, enabling data parallel processing among a heterogeneous mix of CPUs and GPUs by managing subbuffers and tracking updates.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If separate programming environments are used for CPUs and GPUs, then each compute unit can be optimized for its specific architecture, but it becomes difficult to leverage processing resources from both CPUs and GPUs in a unified application
Solution Approach 1:
The patent implements a universal buffer interface that can be accessed by multiple compute units with different architectures (CPUs, GPUs, FPGAs). The buffer object serves multiple functions: storing data for processing, sharing data across heterogeneous compute units, and providing a standardized access interface. This allows a single application to leverage resources from different compute unit types without requiring separate programming environments for each.
2Ease of operation
If data is copied between CPU memory and GPU memory for processing, then data can be accessed by the appropriate compute unit, but memory bandwidth is consumed and data transfer time is increased
Solution Approach 1:
The system performs preliminary actions by pre-copying data to the compute unit's memory before processing is needed, and pre-tracking which compute units have modified data in the buffer. This allows the system to proactively manage data transfers based on anticipated processing needs rather than reacting to actual access requirements, reducing unnecessary data transfer time.
Solution Approach 2:
The patent implements a feedback mechanism where the system tracks which compute units have modified data in the buffer and uses this information to determine when and what data to copy back. This feedback-driven approach ensures that data is transferred only when necessary, minimizing bandwidth consumption and transfer time while maintaining data consistency across heterogeneous compute units.
3Ease of manufacture
If vendor-specific interfaces are required for each GPU, then GPU functionality can be fully utilized, but applications cannot leverage processing resources from GPUs of different vendors or from CPUs
Solution Approach 1:
The patent introduces a buffer object as an intermediary layer between applications and heterogeneous compute units. This buffer interface abstracts the vendor-specific details of different GPUs and CPUs, providing a unified access mechanism. The buffer object manages data storage, tracking, and transfer operations, allowing applications to interact with diverse compute units through a single standardized interface while still leveraging each device's specific capabilities.
Data Source
AI summary
A method and an apparatus for a parallel computing program using subbuffers to perform a data processing task in parallel among heterogeneous compute units are described. The compute units can include a heterogeneous mix of central processing units (CPUs) and graphic processing units (GPUs). A system creates a subbuffer from a parent buffer for each of a plurality of heterogeneous compute units. If a subbuffer is not associated with the same compute unit as the parent buffer, the system copies data from the subbuffer to memory of that compute unit. The system further tracks updates to the data and transfers those updates back to the subbuffer.