Convolution Gradient Kernels Using Forward-Pass Reformulation

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Convolution operations in neural networks, particularly the backwards pass, are resource-intensive and unoptimized, leading to significant delays in training due to unoptimized vector operations and memory constraints.

Innovation Solution

A system and method for accelerating convolution operations by reformulating data formats to utilize optimized convolution functional units, employing a task scheduler, convolution functional units, and data converters to perform transformations, allowing for efficient backward convolution calculations.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If the Col2Im routine is used for backwards pass convolution, then the operation can be performed, but the performance is slow due to unoptimized vector operations

Engineering Contradiction:

Improvebackwards pass convolution speedVSAvoidoptimization difficulty

Core Design Contradiction:

ProductivityVSEase of manufacture

Solution Approach 1:

The patent inverts the traditional Col2Im approach by formulating the backwards pass as a forward pass operation. Instead of scattering gradients from matrix to image, the method transposes the gradient matrix and performs a forward convolution with transposed weights, transforming an unoptimized inverse operation into an optimized forward operation that can leverage the same efficient convolution kernels.

Inventive Principle:

Principle #13The other way round (Inversion)

Solution Approach 2:

The patent changes key parameters of the operation by transposing the gradient matrix and weight matrices, and by changing the role of input and output. The transposed gradient matrix becomes the new input, transposed weights become the new filter, and the result is the transposed input matrix. This parameter transformation allows the use of optimized forward convolution routines instead of unoptimized inverse routines.

Inventive Principle:

Principle #35Parameter changes

2Productivity

If FFT is used for convolution operation, then the operation can be performed, but large amounts of memory are required which is difficult on systems with limited memory or memory bandwidth

Engineering Contradiction:

Improveconvolution operation capabilityVSAvoidmemory requirement

Core Design Contradiction:

ProductivityVSQuantity of substance

Solution Approach 1:

The patent changes the computational approach from frequency domain (FFT) to spatial domain direct convolution. By formulating the backwards pass as a forward pass with transposed parameters, the method enables use of efficient spatial domain convolution kernels that have lower memory requirements compared to FFT-based methods, while maintaining computational correctness.

Inventive Principle:

Principle #35Parameter changes

3Productivity

If Im2Col transformation is used for forward pass, then the convolution can be converted to GEMM operation, but the backwards pass requires unoptimized Col2Im routine leading to significant delay

Engineering Contradiction:

Improveforward pass efficiencyVSAvoidtraining delay

Core Design Contradiction:

ProductivityVSLoss of time

Solution Approach 1:

The patent inverts the backwards pass operation to match the forward pass structure. Instead of using Col2Im to revert the Im2Col transformation, the method formulates the backwards pass as a forward convolution with transposed parameters, allowing the same Im2Col-based GEMM optimization to be applied to both forward and backward passes, thereby eliminating the performance bottleneck.

Inventive Principle:

Principle #13The other way round (Inversion)

Solution Approach 2:

The patent makes the forward pass convolution routine universal by showing that the same optimized forward convolution kernel can be used for both forward propagation and backward propagation, merely by transposing the input matrices and weights. This eliminates the need for separate optimized routines and allows the highly optimized forward pass code to handle both directions of computation.

Inventive Principle:

Principle #6Universality (Multi-functionality)

Data Source

PatentUS12578923B2Method and apparatus for generating architecture specific convolution gradient kernels

Publication Date: 2026.03.17 HUAWEI TECH CO LTD

AI summary

A method for accelerating a convolution operation includes receiving from an I/O interface, a first data set and a second data set. Transforming the first data set into a first converted data set, the first converted data set having the first format. Transforming the second data set into a second converted data set, the second converted data set having the second format. Loading into a convolution functional unit, the first converted data set and the second converted data set, where the convolution functional unit is configured to receive a first data in a first format, to receive a second data in a second format, and to output a third data in a third format. Receiving, by the task scheduler from the convolution functional unit, a result in the third format.