Task Scheduling for Non-Coherent Devices
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Systems with non-coherent devices face challenges in maintaining data coherence, particularly with GPUs, where immediate consistency operations reduce latency but increase bandwidth costs, while batch processing reduces updates but increases latency, and there's a need to minimize latency without affecting bandwidth.
Innovation Solution
A method for scheduling tasks by determining necessary consistency operations for each non-dependent task, performing them selectively, and issuing tasks to non-coherent processing devices as soon as possible, allowing subsequent tasks to be processed in parallel, thereby reducing latency and avoiding pipeline bubbles.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If consistency operations are performed immediately for each task, then latency is reduced, but bandwidth cost increases due to multiple updates
Solution Approach 1:
The patent applies partial action by performing only the necessary subset of consistency operations required for each specific task rather than all possible consistency operations. The system determines which consistency operations are actually needed for a given task and performs only those, avoiding unnecessary bandwidth consumption while still reducing latency compared to batch processing.
Solution Approach 2:
The patent segments the consistency operations into task-specific subsets. Instead of treating consistency operations as a monolithic batch process, the system divides them into individual operations needed for each task, allowing selective execution and parallel processing of different task-specific consistency operations.
2Loss of energy
If consistency operations are batched together, then bandwidth cost is reduced by avoiding multiple updates, but latency increases
Solution Approach 1:
The patent performs exactly the necessary consistency operations for each task without over-processing. By determining the minimal required subset of consistency operations for each task and performing them immediately when ready, the system avoids the latency penalty of batch processing while preventing bandwidth waste from excessive operations.
Solution Approach 2:
The system performs consistency operations as preliminary actions immediately before each task execution when they are ready, rather than delaying them until a batch is complete. This preliminary execution of necessary consistency operations eliminates waiting time in the pipeline while maintaining bandwidth efficiency through selective operation performance.
3Reliability
If all consistency operations are performed before task execution, then data coherence is ensured, but pipeline bubbles are introduced reducing productivity
Solution Approach 1:
The patent performs only the necessary subset of consistency operations for each specific task rather than all possible consistency operations. This partial approach ensures data coherence for the required data while avoiding pipeline stalls that would result from waiting for complete consistency operation sets, thereby maintaining pipeline efficiency.
Solution Approach 2:
The system segments consistency operations into task-specific requirements and processes them independently. This segmentation allows the GPU pipeline to continue processing other tasks while consistency operations for specific tasks are being performed, preventing pipeline bubbles and maintaining productivity while ensuring coherence where needed.
4Speed
If tasks are issued to non-coherent devices with local storage, then processing speed is improved, but data coherence maintenance becomes more complex
Solution Approach 1:
The patent simplifies coherence management by performing only the necessary consistency operations for each task rather than implementing complex comprehensive coherence protocols. This approach maintains data coherence for the specific data needed by each task while avoiding the complexity of managing full system-wide coherence, enabling faster processing on devices with local storage.
Solution Approach 2:
The system acts as an intermediary by determining and performing the specific consistency operations needed before tasks are issued to non-coherent devices. This intermediary role simplifies the coherence management burden on the processing devices themselves, allowing them to operate at full speed with local storage while the system handles the necessary coherence maintenance.
Data Source
AI summary
A method for scheduling tasks to be processed by one of a plurality of non-coherent processing devices, at least two of the devices being heterogeneous devices and at least some of said tasks being targeted to a specific one of the processing devices. The devices process data that is stored in local storage and in a memory accessible by at least some of the devices. The method includes the steps of: for each of a plurality of non-dependent tasks to be processed by the device, determining consistency operations required to be performed prior to processing the non-dependent task; performing the consistency operations for one of the non-dependent tasks and on completion issuing the task to the device for processing; performing consistency operations for a further non-dependent task such that, on completion of the consistency operations, the device can process the further task.