Hardware Accelerator On-Chip Padding for Convolution

Solution Overview

Problem

Traditional convolution operations in artificial neural networks require significant data movement, consuming power and resources and introducing delays due to the need for padding data across the memory hierarchy.

Innovation Solution

The system notifies a hardware accelerator of the input matrix boundaries, allowing it to generate padding data locally, thereby eliminating the need to transfer padding data across the memory hierarchy, and performs convolution operations by processing the activation data set and padding data when the filter kernel slides across the boundaries.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If padding data is transferred across the memory hierarchy to support convolution operations, then the convolution operation can be performed correctly, but data movement increases, consuming more power and introducing delays

Engineering Contradiction:

Improveconvolution operation correctnessVSAvoiddata movement time

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The system pre-loads only the necessary activation data into the on-chip buffer before the convolution operation begins, rather than transferring all padding data in advance. The buffer is then configured with boundary information to generate padding data on-demand during the convolution process, eliminating the need to pre-transfer unnecessary padding data across the memory hierarchy.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The invention extracts and separates the padding data generation function from the main data transfer process. By implementing a dedicated padding generation mechanism within the buffer that uses boundary information to create padding data locally, the system removes the need to transfer padding data through the memory hierarchy, keeping only the essential activation data in the transfer path.

Inventive Principle:

Principle #2Taking out (Extraction)

2Reliability

If padding data is transferred across the memory hierarchy to support convolution operations, then the convolution operation can be performed correctly, but power consumption increases

Engineering Contradiction:

Improveconvolution operation correctnessVSAvoidpower consumption

Core Design Contradiction:

ReliabilityVSUse of energy by moving object

Solution Approach 1:

The system pre-loads only the necessary activation data into the on-chip buffer before the convolution operation begins, rather than transferring all padding data in advance. The buffer is then configured with boundary information to generate padding data on-demand during the convolution process, eliminating the need to pre-transfer unnecessary padding data across the memory hierarchy.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The invention extracts and separates the padding data generation function from the main data transfer process. By implementing a dedicated padding generation mechanism within the buffer that uses boundary information to create padding data locally, the system removes the need to transfer padding data through the memory hierarchy, keeping only the essential activation data in the transfer path.

Inventive Principle:

Principle #2Taking out (Extraction)

3Measurement precision

If traditional convolution operations are performed with full data transfer, then computational accuracy is maintained, but data movement and resource consumption increase

Engineering Contradiction:

Improvecomputational accuracyVSAvoiddata movement volume

Core Design Contradiction:

Measurement precisionVSQuantity of substance

Solution Approach 1:

The system pre-loads only the necessary activation data into the on-chip buffer before the convolution operation begins, rather than transferring all padding data in advance. The buffer is then configured with boundary information to generate padding data on-demand during the convolution process, eliminating the need to pre-transfer unnecessary padding data across the memory hierarchy.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The buffer performs self-service by generating its own padding data locally using boundary information and a padding generation mechanism. This self-contained approach allows the buffer to maintain computational accuracy without relying on external data transfers, reducing the overall data movement volume while preserving the integrity of the convolution operation.

Inventive Principle:

Principle #25Self-service

Data Source

PatentUS11699081B2Systems and methods for reducing data movement during convolution operations in artificial neural networks

Publication Date: 2023.07.11 META PLATFORMS INC

AI summary

The disclosed computer-implemented method may include (1) receiving, at a hardware accelerator that supports an ANN, an activation data set that is to undergo a convolution operation via a filter kernel of the ANN, (2) receiving, at the hardware accelerator, an argument indicating that the filter kernel exceeds at least one boundary of the activation data set when slid across a certain position during the convolution operation, (3) determining, based at least in part on the argument, that the hardware accelerator is to generate padding data at the boundary of the activation data set in connection with the certain position of the filter kernel, and then (4) performing, at the hardware accelerator, the convolution operation by processing a portion of the activation data set and the padding data when the filter kernel slides across the certain position. Various other systems and methods are also disclosed.