Dual Command Queue Scheduling for Obstructed Compute Commands
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing command execution methods in computing apparatuses, such as GPUs and CPUs, face inefficiencies due to obstructed commands leading to idle processing elements and the need for additional hardware and computation cycles, especially when handling large numbers of controllers.
Innovation Solution
Implementing a dual command queue system where commands are stored in separate queues with identifiers, allowing out-of-order execution without additional hardware by selecting unobstructed commands for processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If out-of-order execution and pre-emption processes are deployed to handle obstructed commands, then command execution efficiency is improved, but device complexity and computation requirements increase
Solution Approach 1:
The command queue is divided into multiple independent queues (first command queue and second command queue), each handling specific subsets of commands. This segmentation allows the array controller to select from multiple queues without requiring complex polling or pre-emption logic, reducing device complexity while maintaining efficient command execution.
Solution Approach 2:
Commands are pre-sorted and distributed into different queues based on their execution requirements and dependencies before execution begins. This preliminary organization eliminates the need for complex real-time pre-emption processes, as the array controller simply needs to select from pre-organized queues rather than dynamically managing command sequences.
2Reliability
If polling operations are used to manage command execution, then command obstruction handling is improved, but loss of time and energy increase
Solution Approach 1:
Commands are pre-categorized and placed into different queues based on their execution characteristics and resource requirements before the array controller needs to retrieve them. This preliminary sorting eliminates the need for continuous polling to determine command readiness, as the queue structure already organizes commands by execution priority and dependencies, significantly reducing time and energy losses.
3Productivity
If additional hardware is deployed to handle command queues, then command execution efficiency is improved, but device complexity increases
Solution Approach 1:
The command queue is segmented into multiple simpler queues that can be implemented with basic hardware resources. Each queue handles a specific subset of commands, allowing the system to achieve high throughput by parallel processing without requiring complex hardware architectures. The segmentation approach distributes the workload across multiple simple queue structures rather than one complex centralized queue manager.
Data Source
AI summary
A computing apparatus includes: a set of processing elements configured to execute an active command obtained from a plurality of commands; a first command queue; a second command queue; a bank controller configured to place respective subsets of the plurality of commands into the first and second command queues; and an array controller configured to: (i) select one of the first command queue and the second command queue, (ii) retrieve the active command from the selected one of the first command queue and the second command queue, and (iii) deploy the active command to each of the set of processing elements for execution.