Neural Processing Unit Sub-Kernel Convolution for Lower Latency

Solution Overview

Problem

Existing artificial neural network models, particularly convolutional neural networks, face inefficiencies in transpose convolution operations due to increased latency and unnecessary multiply-and-accumulate (MAC) operations when inserting zeros in feature maps, leading to increased computational load.

Innovation Solution

The method and apparatus divide transpose convolution operations into multiple sub-kernel convolutions, optimizing the process by eliminating the need to calculate zeros and reducing MAC operations.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Ease of manufacture

If transpose convolution operation is performed by inserting zeros in feature map and using general convolution operation, then the operation can be implemented using existing convolution hardware, but latency increases and unnecessary MAC operations are performed

Engineering Contradiction:

Improveimplementation compatibilityVSAvoidcomputational latency

Core Design Contradiction:

Ease of manufactureVSLoss of time

Solution Approach 1:

The patent segments the feature map into multiple regions based on the positions of non-zero elements. By dividing the feature map into regions containing non-zero elements and regions containing only zero elements, the system can selectively apply convolution operations only to relevant regions, avoiding unnecessary computations on zero regions while maintaining compatibility with standard convolution hardware architectures.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent performs convolution operations only on partial regions of the feature map that contain non-zero elements, rather than applying the operation to the entire feature map including zero regions. This partial action approach eliminates unnecessary MAC operations on zero elements while still achieving the complete transpose convolution result through strategic selection of computation regions.

Inventive Principle:

Principle #16Partial or excessive action

2Ease of manufacture

If transpose convolution operation is performed by inserting zeros in feature map and using general convolution operation, then the operation can be implemented using existing convolution hardware, but the amount of unnecessary MAC operation increases

Engineering Contradiction:

Improveimplementation compatibilityVSAvoidcomputational energy consumption

Core Design Contradiction:

Ease of manufactureVSLoss of energy

Solution Approach 1:

The patent segments the feature map into multiple regions based on the positions of non-zero elements. By dividing the feature map into regions containing non-zero elements and regions containing only zero elements, the system can selectively apply convolution operations only to relevant regions, avoiding unnecessary computations on zero regions while maintaining compatibility with standard convolution hardware architectures.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent performs convolution operations only on partial regions of the feature map that contain non-zero elements, rather than applying the operation to the entire feature map including zero regions. This partial action approach eliminates unnecessary MAC operations on zero elements while still achieving the complete transpose convolution result through strategic selection of computation regions.

Inventive Principle:

Principle #16Partial or excessive action

Data Source

PatentUS12443825B2Neural processing unit and method of operation thereof

Publication Date: 2025.10.14 DEEPX CO LTD

AI summary

An operating method for a neural processing unit is provided. The method includes determining, by a controller, that an operation performed in a first convolution layer is a transpose convolution operation, dividing, by the controller, a kernel used for the transpose convolution operation into a plurality of sub-kernels, and performing, by at least one processing element, a convolution operation between an input feature map and each of the plurality of sub-kernels in the first convolution layer.