CUDA Thread Block API for Parallel Scheduling Efficiency

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing computer hardware struggles to efficiently manage the execution of computer programs due to inefficiencies in handling the various structures and dependencies within computer programs, leading to delays and resource wastage.

Innovation Solution

The implementation of an application programming interface (API) that allows for the management of thread blocks, including scheduling policies, dimensions, attributes, and memory sharing, to optimize the execution of CUDA programs on processing resources.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If computer programs are executed using traditional processing resources, then the programs can run, but the execution efficiency is low due to inability to handle program structures and dependencies effectively

Engineering Contradiction:

Improveexecution efficiencyVSAvoidexecution delay

Core Design Contradiction:

ProductivityVSLoss of time

Solution Approach 1:

The patent segments computer programs into multiple thread blocks that can be executed independently or dependently. Each thread block represents a discrete unit of computation that can be scheduled and managed separately, allowing the processing resources to handle program structures and dependencies more effectively, thereby improving execution efficiency and reducing delays.

Inventive Principle:

Principle #1Segmentation

2Productivity

If traditional hardware architecture is used, then the system is simple, but it cannot take into account various aspects of computer program structure leading to inefficiencies

Engineering Contradiction:

Improvecomputational efficiencyVSAvoidhardware architecture complexity

Core Design Contradiction:

ProductivityVSDevice complexity

Solution Approach 1:

The patent introduces a dynamic scheduling mechanism that allows the processing resources to adaptively manage thread blocks based on program structure and dependencies. The scheduler can dynamically allocate and execute thread blocks in optimal sequences, taking into account various program aspects, thereby improving computational efficiency without requiring permanent hardware complexity.

Inventive Principle:

Principle #15Dynamics

3Productivity

If more processing resources are allocated to handle program structures, then execution efficiency improves, but resource consumption increases

Engineering Contradiction:

Improveexecution efficiencyVSAvoidresource consumption

Core Design Contradiction:

ProductivityVSUse of energy by moving object

Solution Approach 1:

The patent enables continuous utilization of processing resources by organizing computations into thread blocks that can be executed in efficient sequences. The scheduling mechanism ensures that processing resources remain continuously engaged with useful computational tasks, minimizing idle time and improving execution efficiency without proportionally increasing resource consumption.

Inventive Principle:

Principle #20Continuity of useful action

Data Source

PatentUS12554534B2Application programming interface to indicate thread blocks

Publication Date: 2026.02.17 NVIDIA CORP

AI summary

Apparatuses, systems, and techniques to execute CUDA programs. In at least one embodiment, an application programming interface is performed to indicate two or more blocks of threads to be scheduled in parallel.