Hardware Accelerator Local Memory Matrix Multiplication Power Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional approaches to performing convolution operations in artificial neural networks (ANNs) consume high amounts of computing power and resources, leading to complex system designs and inefficient power usage due to large data transfers and lack of spatial reuse.
Innovation Solution
A hardware accelerator with a local memory device (LMD) and matrix multiplication unit (MMU) executes matrix multiplication operations using filter and activation matrices, reusing values stored in fast, power-efficient memory and translating strides into simple line reads to reduce power consumption and enhance efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional approaches are used to perform convolution operations, then computing power and resources are consumed, but power consumption is high and system design becomes complex
Solution Approach 1:
The system segments convolution operations into distinct matrix multiplication operations that can be executed independently and in parallel. By breaking down the convolution process into separate matrix multiplication tasks, the system achieves better resource utilization and reduced power consumption while maintaining computational efficiency.
Solution Approach 2:
The patent introduces an intermediary layer of matrix multiplication units that act as mediators between the input data and the convolution operation results. This intermediary structure enables more efficient data flow and computation, reducing the overall power consumption while maintaining productivity.
2Productivity
If traditional approaches are used to perform convolution operations, then computing resources are utilized, but data transfers are large and power usage is inefficient
Solution Approach 1:
The system performs preliminary organization of data into matrix formats before the actual convolution operations. By pre-structuring the input data and filters as matrices, the system minimizes data transfer requirements during computation and reduces energy loss associated with data movement between memory and processing units.
Solution Approach 2:
The patent changes the representation parameters of the data by converting traditional convolution inputs into matrix formats. This parameter transformation enables more efficient memory access patterns and reduces the volume of data that needs to be transferred during computation, thereby reducing energy loss.
3Productivity
If traditional approaches are used to perform convolution operations, then computation is performed, but spatial reuse is lacking and system design becomes complex
Solution Approach 1:
The system implements universal matrix multiplication units that can handle multiple convolution operations with different parameters. These multi-functional units eliminate the need for specialized hardware for each convolution task, reducing system design complexity while maintaining high computation speeds through efficient resource sharing.
Solution Approach 2:
The patent uses copying techniques by creating matrix representations of filters and input data that can be reused across multiple convolution operations. This copying approach enables spatial reuse of computational results and reduces the complexity of managing diverse computational resources.
Data Source
AI summary
A computer-implemented method may include retrieving, via a remote data bus from a data store remote from a hardware accelerator to a local memory device (LMD) included in the hardware accelerator, (1) a filter matrix comprising a set of filter vectors corresponding to a filter location included in each of a set of filters of a convolutional layer of an artificial neural network (ANN), and (2) an activation matrix comprising a primary and a secondary set of activation vectors, each activation vector included in an activation volume inputted into the convolutional layer. The method may also include directing a hardware matrix multiplication unit (MMU) included in the hardware accelerator and communicatively coupled to the LMD via a local data bus, to execute a matrix multiplication operation (MMO) using the filter matrix and the activation matrix.