GPU Coprocessor Self-Scheduling for Low Latency Persistent Computing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current GPU architectures and programming models are inefficient for ultra-low power applications that require minimal parallel processing, leading to significant task allocation and scheduling overhead, and fail to provide lower latency and persistent computing for applications with small input data sets and fast real-time processing needs.
Innovation Solution
A GPU coprocessor with a single instruction, multiple data (SIMD) unit that self-schedules sub-tasks based on input data flow events, utilizing inter-lane crossbar and intra-lane vector general purpose register (VGPR) biased indexing mechanisms, allowing for efficient execution of persistent compute kernels and matrix transposition operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional GPU architectures with host CPU dispatching batches of kernels are used, then parallel processing capability is provided, but task allocation and scheduling overhead becomes significant for applications with limited compute ability and minimal parallel processing requirements
Solution Approach 1:
The GPU coprocessor autonomously schedules and executes sub-tasks without requiring continuous host CPU intervention. The single SIMD unit monitors input data flow events and self-schedules sub-tasks, eliminating the overhead of host CPU task allocation and scheduling while maintaining parallel processing efficiency.
Solution Approach 2:
The patent extracts the scheduling function from the host CPU and embeds it within the GPU coprocessor itself. By incorporating an autonomous scheduling mechanism in the GPU, the system removes the bottleneck of host CPU task management while preserving parallel processing capabilities.
2Productivity
If traditional GPU architectures are used for applications with small input data sets, then computation resources are deployed, but latency is increased and real-time reaction capability is reduced
Solution Approach 1:
The GPU coprocessor is pre-configured with persistent compute kernels and maintains a ready state for immediate execution. By pre-loading computation resources and maintaining an autonomous operational mode, the system eliminates startup latency and enables fast real-time reaction to small input data sets.
Solution Approach 2:
The patent implements persistent computing where the GPU coprocessor continuously monitors input data flow and executes computation tasks without interruption. This continuous operational mode eliminates idle periods and reduces latency for real-time processing of small data sets.
3Productivity
If parallel processing architectures with multiple SIMD units are used, then processing width is increased, but power consumption increases for ultra-low power applications
Solution Approach 1:
The patent employs a single SIMD unit that operates at full capacity rather than multiple SIMD units operating at partial capacity. This partial action approach provides sufficient parallel processing capability for ultra-low power applications while minimizing power consumption by avoiding the overhead of multiple processing units.
4Adaptability or versatility
If host CPU is responsible for preparing kernel input data and scheduling tasks, then flexibility in task management is maintained, but processing speed is reduced for applications requiring fast real-time reaction
Solution Approach 1:
The GPU coprocessor autonomously manages input data preparation and task scheduling without host CPU involvement. The single SIMD unit self-schedules sub-tasks based on input data flow events, maintaining operational flexibility while achieving fast real-time processing speeds by eliminating host CPU communication overhead.
Data Source
AI summary
Systems, apparatuses, and methods for implementing a graphics processing unit (GPU) coprocessor are disclosed. The GPU coprocessor includes a SIMD unit with the ability to self-schedule sub-wave procedures based on input data flow events. A host processor sends messages targeting the GPU coprocessor to a queue. In response to detecting a first message in the queue, the GPU coprocessor schedules a first sub-task for execution. The GPU coprocessor includes an inter-lane crossbar and intra-lane biased indexing mechanism for a vector general purpose register (VGPR) file. The VGPR file is split into two files. The first VGPR file is a larger register file with one read port and one write port. The second VGPR file is a smaller register file with multiple read ports and one write port. The second VGPR introduces the ability to co-issue more than one instruction per clock cycle.