Hardware Accelerator Cache Reuse for Neural Network Power Reduction

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Traditional methods for performing convolution operations in artificial neural networks (ANNs) consume high amounts of power and resources, complicating system designs and failing to leverage spatial reuse, leading to inefficient power management and increased data transfer requirements.

Innovation Solution

A hardware accelerator with a local memory device and matrix multiplication unit (MMU) is used to maintain filter and activation matrices in caches, allowing for efficient reuse of values and translation of strides into simple line reads, thereby reducing power consumption and improving computation efficiency.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Use of energy by moving object

If traditional methods are used for performing convolution operations, then computation can be performed, but power consumption is high and resource usage is inefficient

Engineering Contradiction:

Improvepower consumptionVSAvoidcomputation efficiency

Core Design Contradiction:

Use of energy by moving objectVSProductivity

Solution Approach 1:

The patent segments the convolution computation into distinct matrix multiplication operations using filter matrices and activation matrices. By organizing the computation into separate matrix operations that can be cached and reused, the system reduces redundant calculations and improves energy efficiency while maintaining computational productivity.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent implements preliminary action by pre-loading filter matrices and activation matrices into cache memory before computation. This allows the MMU to reuse these matrices multiple times without repeated access to slower external memory, significantly reducing power consumption while maintaining high computation efficiency.

Inventive Principle:

Principle #10Preliminary action

2Quantity of substance

If traditional convolution methods are used, then computation is performed, but data transfer requirements increase and memory organization becomes complex

Engineering Contradiction:

Improvedata transferVSAvoidmemory organization

Core Design Contradiction:

Quantity of substanceVSDevice complexity

Solution Approach 1:

The patent merges the filter storage and activation storage into a unified cache memory structure that holds both filter matrices and activation matrices. This consolidation reduces the overall memory organization complexity and minimizes data transfer requirements by keeping all necessary matrices locally available during computation.

Inventive Principle:

Principle #5Merging (Combining)

Data Source

PatentUS11599181B1Systems and methods for reducing power consumption of convolution operations of artificial neural networks

Publication Date: 2023.03.07 META PLATFORMS INC

AI summary

A computer-implemented method may include (1) maintaining (a) a filter matrix in a filter cache included in a local memory device (LMD) included in a hardware accelerator, and (b) a plurality of activation matrices corresponding to different rows of an activation volume in an activation cache included in the LMD, (2) for each activation matrix, directing a matrix multiplication unit (MMU) included in the hardware accelerator to execute a matrix multiplication operation (MMU) using the filter matrix and the activation matrix, (3) loading an additional filter matrix into the filter cache, and (4) directing the MMU to execute a plurality of additional MMOs, each additional MMO using one filter matrix included in the filter cache and one activation matrix included in the activation cache, such that the MMU reuses the filter matrix for at least one additional MMO and uses the additional filter matrix for a different additional MMO.