CUDA Block Cluster Memory Sharing for Thread Synchronization

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing computer systems struggle to efficiently manage and synchronize the execution of computer programs due to inefficiencies in processing resources, particularly in handling various structures of computer programs without considering their dependencies, leading to delays and resource wastage.

Innovation Solution

The implementation of an application programming interface (API) that allows for the management of block clusters, enabling the scheduling, synchronization, and sharing of resources among groups of threads, including the use of block clusters to execute software kernels, and the indication of scheduling policies and attributes to optimize thread execution.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If traditional processing resources execute computer programs without considering program structures and dependencies, then processing can be performed, but execution efficiency deteriorates due to delays and resource wastage

Engineering Contradiction:

Improveexecution efficiencyVSAvoidexecution delays

Core Design Contradiction:

ProductivityVSLoss of time

Solution Approach 1:

The patent segments computer programs into blocks of threads that can be independently scheduled and executed. Each block represents a divisible unit of work that can be processed separately, allowing the system to optimize execution by handling individual blocks rather than treating the entire program as a monolithic structure. This segmentation enables parallel processing and reduces execution delays.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent implements dynamic scheduling of thread blocks based on program structures and dependencies. The system adaptively adjusts the execution order and allocation of thread blocks according to real-time conditions and inter-block dependencies, rather than following a static execution sequence. This dynamic approach optimizes resource utilization and reduces unnecessary waiting time.

Inventive Principle:

Principle #15Dynamics

2Adaptability or versatility

If processing resources handle various computer program structures without considering dependencies, then all programs can be executed, but resource utilization deteriorates leading to wastage

Engineering Contradiction:

Improveprogram structure handlingVSAvoidresource utilization

Core Design Contradiction:

Adaptability or versatilityVSUse of energy by moving object

Solution Approach 1:

The patent employs feedback mechanisms where the system continuously monitors program structures, dependencies, and execution states. Based on this feedback, the scheduler dynamically adjusts the allocation and execution of thread blocks to optimize resource utilization. The system learns from execution patterns and adapts its scheduling decisions to minimize resource wastage while maintaining versatility in handling different program structures.

Inventive Principle:

Principle #23Feedback

Solution Approach 2:

The patent changes execution parameters such as thread block allocation, scheduling priorities, and resource assignment based on detected program structures and dependencies. By dynamically adjusting these parameters according to the specific characteristics of each program, the system achieves both high adaptability to various program structures and optimized resource utilization.

Inventive Principle:

Principle #35Parameter changes

3Ease of operation

If traditional systems execute threads without sophisticated scheduling, then execution is simple, but synchronization efficiency deteriorates causing delays

Engineering Contradiction:

Improveexecution simplicityVSAvoidsynchronization efficiency

Core Design Contradiction:

Ease of operationVSProductivity

Solution Approach 1:

The patent introduces an intermediary scheduling layer that manages thread block execution and synchronization. This intermediary scheduler acts as a mediator between the simple execution model and the complex synchronization requirements, coordinating thread blocks to maintain both operational simplicity and synchronization efficiency. The scheduler handles dependency management and coordination without requiring changes to the underlying execution simplicity.

Inventive Principle:

Principle #24Intermediary (Mediator)

Data Source

PatentUS12578993B2Application programming interface to share memory between groups of blocks of threads

Publication Date: 2026.03.17 NVIDIA CORP

AI summary

Apparatuses, systems, and techniques to execute CUDA programs. In at least one embodiment, an application programming interface is performed to cause memory to be shared between two or more groups of blocks of threads.