GP-GPU Convolution Library via Matrix Expansion

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing convolutional neural network implementations on GP-GPUs face inefficiencies in processing and memory usage, particularly in matrix operations, which hinder performance and portability across different GPU generations.

Innovation Solution

A three-step approach is employed: decoupling the convolution process into row and column parts, expanding the row part into a larger matrix, performing matrix multiplication using CUBLAS, and reducing the output matrix, which significantly reduces memory usage and overhead while maintaining high performance.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Quantity of substance

If conventional convolutional neural network implementations are used on GP-GPUs, then processing can be performed, but memory usage is high and performance is inefficient

Engineering Contradiction:

Improvememory usageVSAvoidprocessing performance

Core Design Contradiction:

Quantity of substanceVSProductivity

Solution Approach 1:

The convolution operation is segmented into three distinct steps: matrix expansion, matrix multiplication, and matrix reduction. This segmentation allows each step to be optimized independently and enables efficient memory management by processing data in manageable chunks rather than loading entire datasets at once.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent transforms the convolution operation from a 2D pixel-wise operation into a matrix operation that processes data in a different dimensional representation. By expanding the row part into a larger matrix and performing matrix multiplication, the system achieves more efficient memory access patterns and computation throughput.

Inventive Principle:

Principle #17Another dimension (Dimensionality change)

2Adaptability or versatility

If existing convolutional neural network libraries are used, then functionality is provided, but portability across different GPU generations is poor

Engineering Contradiction:

ImproveportabilityVSAvoidprocessing performance

Core Design Contradiction:

Adaptability or versatilityVSProductivity

Solution Approach 1:

The library is designed with universal interfaces that can operate across different GPU architectures and generations. By using standard matrix multiplication operations that can be implemented through CUBLAS and other mature libraries, the system achieves both portability and high performance across diverse GPU platforms.

Inventive Principle:

Principle #6Universality (Multi-functionality)

3Device complexity

If complex convolution operations are performed directly, then processing completeness is maintained, but overhead is high

Engineering Contradiction:

Improveoperation complexityVSAvoidprocessing overhead

Core Design Contradiction:

Device complexityVSLoss of time

Solution Approach 1:

The patent replaces the direct mechanical execution of complex convolution operations with a simplified three-step matrix-based approach. This substitution reduces computational overhead by leveraging highly optimized matrix multiplication algorithms and data structures, thereby reducing processing time while maintaining functional completeness.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Data Source

PatentUS9904976B2High performance portable convulational neural network library on GP-GPUs

Publication Date: 2018.02.27 NEC CORP

AI summary

Systems and methods are disclosed for speeding up a computer having a graphics processing unit (GPU) and a general purpose processor (GP-GPU) by decoupling a convolution process for a first matrix into a row part and a column part; expanding the row part into a second matrix; performing matrix multiplication using the second matrix and a filter matrix; and performing reduction on an output matrix.