Parallel Machine Learning Accelerator Architecture for Low Power
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep learning workloads, particularly those involving complex convolutional neural network (CNN) and general matrix-matrix multiplication operations, face challenges in reducing power consumption and latency when operating with lower precision data, as existing hardware accelerators struggle to efficiently perform these operations while maintaining accuracy.
Innovation Solution
A hardware accelerator is designed with distinct sets of processing elements for matrix multiplication, sum of weights and offset multiply, and sum of inputs and offset multiply computations, allowing these operations to be performed in parallel, optimized for a graphics processing unit (GPU) architecture.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If complex convolutional neural network and general matrix-matrix multiplication operations are performed on quantized inputs using existing hardware accelerators, then manufacturing precision and reliability are maintained, but power consumption increases and processing speed decreases
Solution Approach 1:
The hardware accelerator is segmented into distinct functional units: a first set of processing elements dedicated to matrix multiplication, a second set for sum of weights and offset multiply, and a third set for sum of inputs and offset multiply. This segmentation allows each unit to be optimized for its specific function, improving overall efficiency and reducing power consumption while maintaining computation accuracy through specialized hardware design.
Solution Approach 2:
The patent transitions from software-based multi-pass computation to hardware-based parallel processing, adding a spatial dimension to computation. By implementing distinct sets of processing elements that operate simultaneously in parallel, the system achieves higher throughput and lower power consumption per operation compared to sequential software execution.
2Measurement precision
If complex convolutional neural network and general matrix-matrix multiplication operations are performed on quantized inputs using existing hardware accelerators, then manufacturing precision and reliability are maintained, but processing speed decreases and latency increases
Solution Approach 1:
The processing architecture is divided into specialized functional units that can operate concurrently. The first set of processing elements handles matrix multiplication while the second and third sets handle offset multiply operations and input summation simultaneously, eliminating sequential bottlenecks and improving processing speed without compromising accuracy.
Solution Approach 2:
Multiple sets of processing elements operate continuously and in parallel, ensuring that computation proceeds without idle cycles. The pipelined architecture allows different stages of computation to overlap in time, maintaining continuous useful action and reducing overall latency while preserving computation precision.
3Productivity
If distinct sets of processing elements are used for matrix multiplication, sum of weights and offset multiply, and sum of inputs and offset multiply computations, then productivity and processing speed improve, but device complexity increases
Solution Approach 1:
Each set of processing elements is designed with universal functionality to handle multiple operations within its domain. The processing elements can perform various arithmetic operations required by different neural network layers, reducing the need for entirely separate specialized hardware for each operation type and managing complexity while maintaining high productivity.
Solution Approach 2:
The hardware architecture uses configurable parameters to adapt processing elements for different computational tasks. By changing operational parameters rather than physical structure, the system achieves versatility across different neural network operations, improving productivity without proportionally increasing hardware complexity.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
An apparatus to facilitate acceleration of machine learning operations is disclosed. The apparatus comprises accelerator circuitry including a first set of processing elements to perform first computations including matrix multiplication operations, a second set of processing elements to perform second computations including sum of elements of weights and offset multiply operations and a third set of processing elements to perform third computations including sum of elements of inputs and offset multiply operations, wherein the second and third computations are performed in parallel with the first computations.