Multi-dimensional Thread Grouping for CPU-GPU Resource Partitioning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data processing systems struggle to leverage processing resources from both CPUs and GPUs effectively, as applications are typically designed for specific vendors and cannot concurrently utilize resources from multiple processors, limiting their ability to perform data parallel computations across different processing units.
Innovation Solution
The method determines the optimal number of threads to execute on GPUs and CPUs through an API request, partitioning them into thread groups to optimize resource usage, allowing applications to execute concurrently across multiple processing units, including GPUs and CPUs from different vendors.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If applications are designed for specific vendors using vendor-specific interfaces, then they can execute on specific GPUs, but they cannot leverage processing resources of CPUs or GPUs from different vendors
Solution Approach 1:
The patent implements a universal programming interface that allows applications to execute on multiple types of processors (CPUs and GPUs from different vendors) through a common API. The system translates vendor-specific operations into processor-agnostic instructions, enabling one application to leverage resources from any available processor without vendor-specific customization.
Solution Approach 2:
The patent introduces an intermediary layer (runtime system and translation layer) between the application and the physical processors. This intermediary translates high-level thread group operations into vendor-specific instructions, allowing the application to remain vendor-agnostic while efficiently utilizing resources from different processor types and vendors.
2Productivity
If applications use separate programming environments for GPUs and CPUs, then they can be optimized for each device, but it becomes very difficult to leverage processing resources of both CPUs and GPUs concurrently
Solution Approach 1:
The patent merges the separate programming environments for CPUs and GPUs into a unified programming model. By combining thread group management and execution control into a single API framework, developers can write one application that automatically utilizes both CPU and GPU resources without maintaining separate codebases for each processor type.
Solution Approach 2:
The unified programming interface serves multiple functions simultaneously: it manages thread groups for both CPUs and GPUs, handles vendor-specific translations, and coordinates concurrent execution across different processor types, eliminating the need for separate programming environments.
3Productivity
If a single source code is compiled for multiple target processing units, then resource utilization can be optimized, but determining optimal thread group sizes becomes complex across different processor architectures
Solution Approach 1:
The runtime system automatically determines optimal thread group sizes for different target processors by analyzing processor capabilities and workload characteristics. This self-service approach eliminates the need for manual optimization by developers, as the system adapts thread group configurations automatically based on the specific processor architecture being used.
Solution Approach 2:
The system dynamically adjusts thread group size parameters based on the target processor type and vendor. By changing these parameters automatically at runtime based on detected processor characteristics, the system optimizes resource efficiency without requiring complex manual configuration or deep knowledge of each processor architecture.
Data Source
AI summary
A method and an apparatus that partition a total number of threads to concurrently execute executable codes compiled from a single source for target processing units in response to an API (Application Programming Interface) request from an application running in a host processing unit are described. The total number of threads is based on a multi-dimensional value for a global thread number specified in the API. The target processing units include GPUs (Graphics Processing Unit) and CPUs (Central Processing Unit). Thread group sizes for the target processing units are determined to partition the total number of threads according to either a dimension for a data parallel task associated with the executable codes or a dimension for a multi-dimensional value for a local thread group number. The executable codes are loaded to be executed in thread groups with the determined thread group sizes concurrently in the target processing units.