GPU Delegator With Indirect-Pipe Kernel Coordination for Neural Networks

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

The challenge of reducing data processing delays and unnecessary interactions between a central processing unit (CPU) and a graphics processing unit (GPU) during neural network processing in highly integrated semiconductor technology is addressed.

Innovation Solution

A delegator is introduced to manage and delegate interactions between kernels within the GPU, utilizing an indirect-pipe to store execution result addresses and reduce direct CPU-GPU interactions, thereby optimizing neural network processing.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If data transmission occurs in every kernel between CPU and GPU, then neural network processing can be performed, but processing delay increases

Engineering Contradiction:

Improveneural network processing speedVSAvoiddata processing delay

Core Design Contradiction:

ProductivityVSLoss of time

Solution Approach 1:

The patent introduces an indirect-pipe as an intermediary data structure that stores addresses of execution result data. This allows kernels to access each other's results directly through the indirect-pipe without requiring CPU intervention for every data transmission, thereby reducing processing delay while maintaining correct data flow in the neural network pipeline.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Ease of operation

If direct CPU-GPU interactions occur for every kernel, then data can be transmitted, but interaction frequency increases

Engineering Contradiction:

Improvedata transmission capabilityVSAvoidCPU-GPU interaction frequency

Core Design Contradiction:

Ease of operationVSDevice complexity

Solution Approach 1:

The patent extracts the data transmission management function from the CPU-GPU interaction channel and places it within the GPU's command processor. The delegator component, integrated into the command processor, manages indirect-pipe creation and kernel delegation internally, eliminating the need for repeated CPU-GPU interactions for each kernel's data transmission.

Inventive Principle:

Principle #2Taking out (Extraction)

3Reliability

If multiple kernels are executed sequentially with CPU intervention, then processing can be controlled, but processing speed decreases

Engineering Contradiction:

Improvekernel execution controlVSAvoidkernel execution speed

Core Design Contradiction:

ReliabilityVSSpeed

Solution Approach 1:

The patent implements preliminary action by having the delegator pre-create indirect-pipes and pre-establish data dependency relationships between kernels before execution. The command processor uses this pre-configured information to automatically delegate subsequent kernels without CPU intervention, maintaining reliable execution control while significantly increasing execution speed through batch processing.

Inventive Principle:

Principle #10Preliminary action

Data Source

PatentUS12354181B2Graphics processing unit including delegator and operating method thereof

Publication Date: 2025.07.08 SAMSUNG ELECTRONICS CO LTD

AI summary

A graphics processing unit (GPU) configured to process a neural network is provided. The GPU includes: a graphics core configured to execute a plurality of kernels; and a delegator connecting an input and an output of the plurality of kernels based on dependency between the plurality of kernels, and configured to output an event indicating a result of a neural network process.