Kernel Scheduling Cohorts to Minimize Cache Contention
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing computing systems face challenges in efficiently scheduling kernels for execution on integrated circuits, particularly in balancing high throughput and minimizing cache contention in the last level shared cache structure.
Innovation Solution
A computing system with a scheduler that groups kernels into cohorts based on shared data sets, delays scheduling of some cohorts to minimize cache contention, and uses completion time estimates to determine when to schedule next cohorts for execution.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If kernels are scheduled for execution on multiple compute circuits, then throughput is improved, but cache contention increases
Solution Approach 1:
The patent segments kernels into cohorts based on their data access patterns and cache usage characteristics. By grouping kernels with similar cache requirements together, the system can schedule cohorts in a way that minimizes cache contention while maintaining high throughput. Each cohort is treated as a unit of execution, allowing the scheduler to optimize cache utilization across multiple compute circuits.
Solution Approach 2:
The scheduler dynamically adjusts kernel scheduling decisions based on real-time cache status and compute circuit availability. It monitors cache contention levels and adapts the scheduling policy to balance throughput optimization with cache contention minimization, rather than using a static scheduling approach.
2Speed
If available bandwidth for lower levels of memory hierarchy is saturated, then data access speed is improved, but response bandwidth inefficiency remains
Solution Approach 1:
The system implements feedback mechanisms that monitor cache hit rates, memory access patterns, and compute circuit performance. This feedback information is used to dynamically adjust scheduling decisions, ensuring that data access speed is optimized while response bandwidth efficiency is maintained through adaptive resource allocation.
Solution Approach 2:
The scheduler changes scheduling parameters such as cohort priority, execution timing, and resource allocation based on observed system performance. By adjusting these parameters dynamically, the system optimizes both data access speed and response bandwidth efficiency according to actual workload characteristics.
Data Source
AI summary
An apparatus and method for efficiently scheduling kernels for execution in a computing system. In various implementations, a computing system includes a cache and a processing circuit with multiple compute circuits and a scheduler. The scheduler groups kernels into scheduling groups where each scheduling group includes particular kernels of the multiple kernels that access a same data set different from a data set of another scheduling group. Each of these scheduling groups is referred to as a “cohort.” The scheduler accesses completion time estimates of kernels of the cohorts. Using the completion time estimates, the number of kernels currently executing, and the number of remaining kernels that have not yet begun execution of each currently scheduled cohort, the scheduler determines whether to immediately schedule a next cohort or delay scheduling the next cohort. By doing so, the scheduler balances throughput and cache contention.