Hardware Accelerator Pre-configured Matrix Transform Coefficients
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Neural networks, particularly convolutional neural networks, are computationally intensive, leading to significant computational and energy burdens, despite efforts to reduce arithmetic operations through domain-transformation-based algorithms like FFT and Winograd minimal filtering algorithms.
Innovation Solution
A special-purpose hardware accelerator is pre-configured with coefficients for performing domain-transform operations in convolution operations, including a cache for input matrices and a matrix-multiplication subsystem with pre-configured coefficients for transforming input and parameter matrices, and performing inverse-transformation operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If domain-transformation-based algorithms (FFT, Winograd) are used to reduce arithmetic operations, then computational efficiency is improved, but device complexity increases due to pre-computed coefficient storage requirements
Solution Approach 1:
The patent pre-computes and stores transformation coefficients (e.g., Winograd coefficients) in the hardware accelerator before runtime. This preliminary action eliminates the need for complex runtime coefficient calculations, reducing operational complexity while maintaining high computational efficiency through direct application of pre-computed values during convolution operations
Solution Approach 2:
The patent creates a specialized hardware copy of the transformation logic with fixed coefficients embedded directly in the accelerator circuitry. This copying approach replicates the essential transformation functionality in hardware rather than software, achieving faster execution with simplified runtime operations by hardcoding the coefficient transformation matrices
2Speed
If more coefficients are pre-configured in the hardware accelerator, then convolution operation speed is improved, but manufacturing cost increases
Solution Approach 1:
The patent applies local quality by selectively pre-configuring only the specific coefficients needed for particular convolution configurations (e.g., fixed kernel sizes like 3x3, 5x5). Rather than hardcoding all possible coefficients for all kernel sizes, the hardware accelerator is optimized with localized coefficient sets tailored to common use cases, reducing manufacturing complexity while maintaining high speed for those specific operations
Solution Approach 2:
The patent incorporates dynamic coefficient loading capabilities that allow the hardware accelerator to adapt between different coefficient sets based on runtime requirements. This dynamic approach enables the system to switch between pre-configured coefficient sets for different kernel sizes or transformation types, achieving high-speed operations across multiple configurations without permanently hardcoding all possible coefficients, thereby controlling manufacturing costs
3Device complexity
If transformation coefficients are stored in memory rather than pre-configured in hardware, then device complexity is reduced, but computational speed decreases due to memory access overhead
Solution Approach 1:
The patent implements a nested memory hierarchy where transformation coefficients are stored in fast on-chip memory (e.g., registers or small SRAM arrays) that is tightly coupled to the computation units. This nested approach places frequently accessed coefficients in the fastest available memory layer close to the arithmetic logic, eliminating slow external memory accesses while keeping the overall device architecture relatively simple through hierarchical organization
Data Source
AI summary
A special-purpose hardware accelerator may include a cache configured to store an input matrix related to performing a convolution operation and a matrix-multiplication subsystem pre-configured with matrix-transform coefficients for performing matrix-transform operations. The matrix-multiplication subsystem may perform the convolution operation by (1) reading the input matrix from the cache, (2) transforming the input matrix via matrix multiplication, (3) transforming, via matrix multiplication, a parameter matrix that includes convolution parameters for performing the convolution operation, (4) applying the transformed parameter matrix to the transformed input matrix via an element-wise multiplication operation, and then (5) performing an inverse-transformation operation on the results of the element-wise multiplication operation to create an output matrix for the convolution operation. Various other systems and methods are also disclosed.