Convolution Transpose Hardware via Filter Segmentation and Interleaving

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Implementing a convolution transpose operation in hardware, such as a neural network accelerator, is inefficient due to the need for calculating intermediate results and accumulating them in separate steps.

Innovation Solution

Divide the filter into a plurality of sub-filters, perform a convolution operation between the input tensor and each sub-filter to generate sub-output tensors, and then interleave the output elements of these sub-output tensors to form the final output tensor for the convolution transpose.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If a convolution transpose operation is implemented in hardware using traditional methods, then the operation can be performed, but it requires calculating intermediate results and accumulating them in separate steps which reduces efficiency

Engineering Contradiction:

Improveoperation efficiencyVSAvoidhardware complexity

Core Design Contradiction:

ProductivityVSDevice complexity

Solution Approach 1:

The filter is divided into multiple sub-filters, and the convolution transpose operation is segmented into multiple parallel convolution operations. Each sub-filter processes a portion of the input tensor simultaneously, eliminating the need for sequential intermediate result calculation and accumulation, thus improving productivity without significantly increasing hardware complexity

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

Multiple convolution operations are merged into a single hardware pipeline by processing them in parallel. The intermediate results from different sub-filters are combined through interleaving operations within the same hardware pass, eliminating the need for separate calculation and accumulation steps, thereby improving operational efficiency

Inventive Principle:

Principle #5Merging (Combining)

2Adaptability or versatility

If a convolution transpose operation is implemented in hardware, then the operation can be performed, but existing hardware designed for convolution layers cannot be directly used

Engineering Contradiction:

Improvehardware reusabilityVSAvoidimplementation complexity

Core Design Contradiction:

Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The same hardware logic designed for convolution operations is made universal by applying it to convolution transpose operations through filter segmentation and sub-filter processing. The hardware can now perform both convolution and convolution transpose operations using the same physical resources, improving adaptability and reducing implementation complexity

Inventive Principle:

Principle #6Universality (Multi-functionality)

Solution Approach 2:

Instead of designing dedicated hardware for convolution transpose operations, the invention inverts the approach by using convolution hardware in reverse - applying convolution operations with segmented filters to achieve transpose convolution effects. This allows existing convolution hardware to be reused for transpose operations, improving hardware versatility

Inventive Principle:

Principle #13The other way round (Inversion)

Data Source

PatentUS20250068693A1Methods and systems for implementing a convolution transpose layer of a neural network

Publication Date: 2025.02.27 IMAGINATION TECH LTD

AI summary

Methods and systems for performing a convolution transpose operation between an input tensor having a plurality of input elements and a filter comprising a plurality of filter weights. The method includes: dividing the filter into a plurality of sub-filters; performing, using hardware logic, a convolution operation between the input tensor and each of the plurality of sub-filters to generate a plurality of sub-output tensors, each sub-output tensor comprising a plurality of output elements; and interleaving, using hardware logic, the output elements of the plurality of sub-output tensors to form a final output tensor for the convolution transpose.