Deep Learning Accelerator Compiler-Configurable Hardware
Solution Overview
Problem
Current integrated circuit devices for Artificial Neural Networks (ANNs) face challenges in reducing energy consumption and computation time, particularly in efficiently executing matrix and vector operations due to limitations in hardware configuration and memory access bottlenecks.
Innovation Solution
The integration of a Deep Learning Accelerator (DLA) with configurable hardware options and a compiler that generates optimized instructions for matrix computations, allowing for autonomous execution of ANNs with reduced energy consumption and computation time by optimizing hardware configurations and improving data access efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If fixed hardware configuration is used for matrix operations, then device complexity is reduced, but adaptability to different ANN workloads deteriorates
Solution Approach 1:
The patent implements dynamic hardware configuration by allowing the processing unit to reconfigure its circuitry between different operational modes (matrix-matrix, matrix-vector, vector-vector operations) based on the specific workload requirements. This enables the hardware to adapt to different ANN workloads while maintaining a relatively simple base architecture, resolving the contradiction between adaptability and device complexity.
Solution Approach 2:
The processing unit is designed with universal circuit configurations that can perform multiple types of operations (matrix-matrix multiplication, matrix-vector multiplication, vector-vector operations) through configurable parameters. This multi-functionality allows a single hardware design to handle diverse ANN workloads without requiring separate specialized circuits for each operation type, thereby improving adaptability without proportionally increasing device complexity.
2Productivity
If more processing units are added to increase computation speed, then productivity is improved, but device complexity and energy consumption worsen
Solution Approach 1:
The patent achieves higher computation speed by changing operational parameters within existing processing units rather than simply adding more units. The configurable hardware options allow optimization of computational parameters such as precision, operation type, and data flow patterns, enabling the same hardware to deliver variable performance levels and improving productivity without linearly increasing device complexity.
Solution Approach 2:
The processing units are designed to perform computations with configurable precision levels and operational depths. By allowing partial computation or early termination of operations when sufficient accuracy is achieved, the system can maintain high productivity for simple workloads while avoiding the need for excessive processing power, thus improving computation speed without proportionally increasing device complexity.
3Speed
If hardware is optimized for specific operations, then computation speed is improved, but adaptability to other operations deteriorates
Solution Approach 1:
The patent implements dynamic reconfiguration capability that allows the processing unit to switch between different operational modes (matrix-matrix, matrix-vector, vector-vector) based on the incoming workload. This dynamic adaptation enables the hardware to be optimized for the current operation type while maintaining the capability to handle other operation types, thus achieving high computation speed for specific operations without sacrificing overall adaptability.
Solution Approach 2:
The processing unit incorporates universal circuit designs that can perform multiple operation types through configurable parameters and control logic. By designing circuits that are inherently capable of multiple functions with appropriate configuration, the system achieves computation speed optimization for each operation type while maintaining versatility across different operation types, resolving the contradiction between speed and adaptability.
4Productivity
If memory access is increased to improve data availability, then computation continuity is improved, but energy consumption worsens
Solution Approach 1:
The patent implements local optimization of data access by configuring processing units to access only the specific data elements needed for current operations, rather than broadly accessing entire memory blocks. The configurable hardware options enable optimization of memory access patterns to match the specific computation requirements, improving computation continuity by ensuring needed data is available while minimizing unnecessary memory accesses that would increase energy consumption.
Solution Approach 2:
The system allows partial memory access where only the necessary portion of data is fetched into the processing unit for current computations. By accessing memory partially and only when needed rather than continuously or excessively, the system maintains computation continuity for critical operations while reducing overall energy consumption associated with memory access, resolving the contradiction between productivity and energy use.
Data Source
AI summary
Systems, devices, and methods related to a Deep Learning Accelerator and memory are described. For example, an integrated circuit device may be configured to execute instructions with matrix operands and configured with random access memory. A compiler can convert a description of an artificial neural network into a compiler output through optimization and/or selection of hardware options of the integrated circuit device. The compiler output can include parameters of the artificial neural network, instructions executable by processing units of the Deep Learning Accelerator to generate an output of the artificial neural network responsive to an input to the artificial neural network, and hardware options to be stored in registers connected to control hardware configurations of the processing units.