Hardware Accelerated Convolution via Virtual GEMM Mapping
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional machine learning algorithms consume excessive memory bandwidth and power due to the conversion of convolution operations into matrix multiplication, resulting in larger image matrices and increased memory access requests.
Innovation Solution
The implementation of a virtual GEMM space in combination with convolutional mapping hardware that maps convolution operations to processors, reducing memory bandwidth by partitioning the result area and using dedicated hardware to accelerate address calculations and padding checks, converting expensive operations into less power-consuming ones.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If convolution operations are converted into matrix multiplication using conventional algorithms, then the convolution can be performed using general-purpose processors, but memory bandwidth consumption increases due to larger image matrices and duplicative data
Solution Approach 1:
The patent segments the convolution operation into distinct functional components: address generation, data retrieval, multiplication, and accumulation. By organizing the computation this way, each processor element can be optimized for specific tasks, reducing redundant memory access while maintaining compatibility with standard processor architectures.
Solution Approach 2:
The patent introduces an intermediary memory structure (such as on-chip buffer memory or register file) that sits between global memory and the processing elements. This intermediary holds duplicative data locally, allowing multiple processors to access the same data without requiring repeated reads from global memory, thus reducing overall memory bandwidth consumption.
2Productivity
If convolution operations are converted into matrix multiplication, then the computation can be parallelized across multiple processors, but power consumption increases due to increased memory access requests
Solution Approach 1:
The patent divides the parallel processing architecture into segments where each processing element has dedicated or shared local storage. This segmentation allows parallel processors to operate simultaneously while accessing data from local rather than global memory, significantly reducing the power cost of memory access while maintaining high parallel productivity.
Solution Approach 2:
The patent implements preliminary action by pre-loading duplicative data into local memory structures before the parallel computation begins. This way, when multiple processors need to access the same data during parallel execution, the data is already available in fast, low-power local memory rather than requiring repeated high-power accesses to global memory.
3Adaptability or versatility
If software is used to perform the mapping process from virtual GEMM space to image space, then flexibility is maintained, but power consumption increases due to additional assembly instructions for address calculations and padding checks
Solution Approach 1:
The patent replaces the software-based mapping mechanism with hardware-based address generation logic. Instead of executing software instructions for address calculation and padding checks, dedicated hardware circuits perform these functions directly, eliminating the power consumption associated with software instruction execution while preserving mapping flexibility through configurable hardware parameters.
Data Source
AI summary
A processing device is provided which includes memory configured to store data and a processor configured to determine, based on convolutional parameters associated with an image, a virtual general matrix-matrix multiplication (GEMM) space of a virtual GEMM space output matrix and generate, in the virtual GEMM space output matrix, a convolution result by matrix multiplying the data corresponding to a virtual GEMM space input matrix with the data corresponding to a virtual GEMM space filter matrix. The processing device also includes convolutional mapping hardware configured to map, based on the convolutional parameters, positions of the virtual GEMM space input matrix to positions of an image space of the image.