Dynamic Thread Block Topology Modification in Kernel Functions

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Conventional computing methods face inefficiencies when executing computations that require changing the topological structure of thread blocks, leading to high overhead, memory bandwidth wastage, data delay, and increased programming complexity.

Innovation Solution

A method that dynamically modifies the size and topological structure of thread blocks within a kernel function by identifying call instructions for thread block modifications, configuring required thread resources, and inserting control instructions to enable relevant computations, reducing the need for multiple kernel function starts and allowing built-in coordinate information for data access.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If the kernel function is started multiple times to change thread block topology, then different computing tasks can be executed with appropriate thread configurations, but the overhead for starting the kernel function increases, memory bandwidth is wasted, and data delay is increased

Engineering Contradiction:

Improvethread block topology adaptabilityVSAvoidcomputing performance

Core Design Contradiction:

Adaptability or versatilityVSProductivity

Solution Approach 1:

The patent implements dynamic modification of thread block topology within a single kernel function execution. The system allows the thread block size and configuration to change dynamically during execution based on computational requirements, rather than requiring multiple separate kernel function starts. This is achieved through runtime thread block modification mechanisms that adjust the topological structure adaptively while maintaining continuous execution flow.

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The patent ensures continuous execution of the kernel function without interruption or restart. By modifying thread block topology dynamically within the same execution context, the system maintains continuous computational flow, avoiding the overhead and data transfer delays associated with stopping and restarting the kernel function multiple times.

Inventive Principle:

Principle #20Continuity of useful action

2Adaptability or versatility

If the kernel function is started multiple times to change thread block topology, then different computing tasks can be executed with appropriate thread configurations, but coding complexity increases and the error rate increases

Engineering Contradiction:

Improvethread block topology adaptabilityVSAvoidcoding ease

Core Design Contradiction:

Adaptability or versatilityVSEase of operation

Solution Approach 1:

The patent implements automatic thread block topology management where the system self-adjusts the thread block configuration based on computational requirements. The compiler or runtime system automatically handles the modification of thread block topology without requiring manual intervention or complex programming from the user, thereby simplifying the coding process while maintaining adaptability.

Inventive Principle:

Principle #25Self-service

3Adaptability or versatility

If thread physical index is used directly for programming without topological structure, then thread block topology can be changed flexibly, but the number of threads loses task assignment topology properties and programming complexity increases

Engineering Contradiction:

Improvethread block topology flexibilityVSAvoidprogramming complexity

Core Design Contradiction:

Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The patent creates a universal thread management mechanism that maintains the benefits of topological organization while allowing flexible configuration changes. The thread block structure serves multiple functions: it provides organized task assignment, enables flexible topology modification, and maintains simplified programming interfaces. This universal approach eliminates the need to choose between fixed topology and raw physical indexing.

Inventive Principle:

Principle #6Universality (Multi-functionality)

Data Source

PatentUS11886846B2Method for executing computation, computing device, computing system, and storage medium

Publication Date: 2024.01.30 SHANGHAI BIREN TECH CO LTD

AI summary

A method for executing computation, a computing device, a computing system, and a storage medium are provided. The method includes: confirming, via a compiler, whether there is a call instruction related to a thread block modification request in a kernel function to be compiled; in response to confirming that there is the call instruction related to the thread block modification request in the kernel function to be compiled, determining a corresponding program segment associated with the call instruction; configuring a required thread block and thread local register for the corresponding program segment; and inserting a control instruction into the corresponding program segment to enable the thread block configured for the corresponding program segment to execute relevant computation of the corresponding program segment, and an unconfigured thread block not to execute the relevant computation. The disclosure can improve overall performance, make coding and maintenance easy and reduce error rate of code.