Neural Network Inference Circuit with Piecewise Linear Activation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing integrated circuits are optimized for dot product computations between activation and weight values, but certain neural networks require other types of calculations, such as activation matrix multiplication and softmax operations, which are not efficiently handled without major hardware modifications.
Innovation Solution
An integrated circuit (IC) with a neural network computation fabric that includes cores for computing dot products, a global channel for aggregation, and post-processing units for various operations, supporting both dot product and non-dot product computations, utilizing piecewise linear activation functions for efficiency and flexibility.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If the IC is optimized for dot product computations, then computational efficiency for dot products is improved, but the ability to perform other neural network computations (activation matrix multiplication, softmax operations) deteriorates
Solution Approach 1:
The neural network computation fabric is designed to perform multiple types of computations including dot products, activation matrix multiplications, and softmax operations using the same hardware resources. The computation units can be dynamically configured to execute different operation types without requiring separate dedicated circuits for each computation type, thereby achieving multi-functionality while maintaining efficiency.
Solution Approach 2:
The computation fabric employs dynamic configuration capabilities where the same hardware units can switch between different computation modes (dot product, matrix multiplication, softmax) based on the neural network layer requirements. This dynamic reconfigurability allows the system to adapt to various computation types while maintaining high computational efficiency for each specific operation.
2Adaptability or versatility
If dedicated hardware is added to support non-dot product computations, then computation versatility is improved, but device complexity and resource usage deteriorate
Solution Approach 1:
Rather than adding separate dedicated hardware units for each computation type, the patent implements a universal computation fabric where existing dot product units can be configured to perform activation matrix multiplications and softmax operations. This approach achieves computation versatility without proportionally increasing hardware complexity, as the same physical resources serve multiple computational purposes.
Solution Approach 2:
The patent merges multiple computation functions into a single unified computation fabric. By combining dot product, matrix multiplication, and softmax capabilities within the same hardware structure, the system avoids the complexity of maintaining separate dedicated circuits for each operation type, thereby reducing overall device complexity while maintaining versatility.
3Adaptability or versatility
If general-purpose computation units are used, then computation versatility is improved, but power consumption and resource usage deteriorate
Solution Approach 1:
The computation fabric dynamically configures its operational mode based on the specific neural network layer being processed. When a dot product operation is required, the units operate in dot product mode with optimized power characteristics; when matrix multiplication or softmax is needed, the same units reconfigure accordingly. This dynamic adaptation allows the system to maintain low power consumption while supporting multiple computation types, avoiding the need for always-on general-purpose units.
Solution Approach 2:
The system changes operational parameters of the computation units to optimize for different computation types. By adjusting configuration parameters rather than physically reconfiguring hardware, the system achieves versatility while minimizing power overhead. The computation units maintain efficient operation across different computation types through parameter optimization rather than requiring high-power general-purpose processing.
Data Source
AI summary
Some embodiments provide a neural network inference circuit for executing a neural network that includes computation nodes. Each respective computation node of a set of the computation nodes includes (i) a respective linear function that includes a respective dot product of input values for the computation node and weight values for the computation node and (ii) a respective non-linear activation function. The neural network inference circuit includes a set of dot product circuits to compute the dot product for a computation node and a post-processing circuit to compute (i) a result of the linear function for the computation node based on the dot product for the computation node and (ii) an output for the computation node by applying a piecewise linear function to the result of the linear function for the computation node to apply the non-linear activation function for the computation node.