Thread Group Scheduler for Parallel Thread Processor

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Conventional multithreaded streaming multiprocessors face inefficiencies when scheduling multiple cooperative thread arrays (CTAs), leading to resource conflicts, idle cycles, and potential deadlocks due to uneven thread completion and resource allocation.

Innovation Solution

A method for scheduling thread groups on a parallel thread processor that identifies available thread groups based on seniority and credit values, allowing simultaneous launch of multiple CTAs while optimizing resource utilization and preventing deadlocks.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If conventional SMP schedules thread groups fairly to ensure each thread group makes equal progress, then thread group fairness is improved, but resource utilization deteriorates when multiple CTAs are executed

Engineering Contradiction:

Improvethread group fairnessVSAvoidresource utilization

Core Design Contradiction:

ReliabilityVSProductivity

Solution Approach 1:

The patent implements dynamic CTA prioritization where the scheduler continuously adjusts priority levels based on execution progress and resource availability. CTAs are assigned dynamic priority values that change over time, allowing the system to adapt to varying workload conditions and optimize both fairness and resource utilization simultaneously

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The scheduler modifies scheduling parameters including priority levels, time slice allocations, and resource distribution based on real-time system state. By changing these parameters dynamically, the system resolves the contradiction between maintaining fair thread group progression and maximizing overall resource utilization across multiple CTAs

Inventive Principle:

Principle #35Parameter changes

2Reliability

If one thread of an executing CTA requires more time to reach a barrier point than other threads, then synchronization correctness is improved, but SMP productivity deteriorates as other CTAs cannot execute

Engineering Contradiction:

Improvesynchronization correctnessVSAvoidSMP execution throughput

Core Design Contradiction:

ReliabilityVSProductivity

Solution Approach 1:

The patent segments the CTA execution into independent thread group units that can be scheduled separately. When one thread within a CTA is slow to reach a barrier, only the affected thread group is delayed while other thread groups from the same or different CTAs can continue executing, preventing system-wide stalls and maintaining high throughput

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The scheduler dynamically adjusts the execution schedule based on real-time thread progress monitoring. When threads reach barrier points at different times, the system dynamically reconfigures which thread groups execute next, allowing parallel execution of independent thread groups while maintaining synchronization correctness for dependent operations

Inventive Principle:

Principle #15Dynamics

3Productivity

If thread groups within an executing CTA perform similar processing operations simultaneously, then computational efficiency is improved, but resource conflicts worsen causing bottlenecks

Engineering Contradiction:

Improvecomputational efficiencyVSAvoidresource conflict management

Core Design Contradiction:

ProductivityVSDevice complexity

Solution Approach 1:

The patent introduces a temporal dimension to resource allocation by implementing time-multiplexed resource sharing. Thread groups that would otherwise conflict for the same resources are scheduled at different time slots, while still achieving computational efficiency through parallel execution of non-conflicting operations across multiple dimensions (spatial threading + temporal scheduling)

Inventive Principle:

Principle #17Another dimension (Dimensionality change)

Solution Approach 2:

The system dynamically changes resource allocation parameters including which functional units are assigned to which thread groups at different time steps. By adjusting these allocation parameters based on current execution needs, the system resolves resource conflicts while maintaining high computational efficiency through optimized resource utilization

Inventive Principle:

Principle #35Parameter changes

4Productivity

If multiple CTAs are launched simultaneously, then overall throughput is improved, but SMP idle time worsens when CTAs exit simultaneously requiring resource reclamation

Engineering Contradiction:

Improveoverall throughputVSAvoidSMP idle time

Core Design Contradiction:

ProductivityVSLoss of time

Solution Approach 1:

The scheduler dynamically launches and terminates CTAs at different times based on workload characteristics and resource availability. By staggering CTA launch times and using asynchronous completion handling, the system maintains continuous pipeline utilization and eliminates idle periods when CTAs would otherwise exit simultaneously, thereby preserving high throughput without sacrificing resource reclamation efficiency

Inventive Principle:

Principle #15Dynamics

Data Source

PatentUS8732713B2Thread group scheduler for computing on a parallel thread processor

Publication Date: 2014.05.20 NVIDIA CORP

AI summary

A parallel thread processor executes thread groups belonging to multiple cooperative thread arrays (CTAs). At each cycle of the parallel thread processor, an instruction scheduler selects a thread group to be issued for execution during a subsequent cycle. The instruction scheduler selects a thread group to issue for execution by (i) identifying a pool of available thread groups, (ii) identifying a CTA that has the greatest seniority value, and (iii) selecting the thread group that has the greatest credit value from within the CTA with the greatest seniority value.