Logarithmic Neural Network Accelerator Circuit
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep neural networks (DNNs) require increasing computational resources and memory, leading to higher hardware demands and optimization challenges, despite relieving human experts of feature engineering burdens.
Innovation Solution
A neural network accelerator is designed to perform operations on a logarithm domain, utilizing a Jacobian logarithmic adder that converts real domain operations into logarithmic additions and subtractions, reducing computational complexity and hardware requirements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If conventional DNN operations are performed on real domain with multipliers, then computation accuracy is maintained, but hardware complexity and computational load increase significantly
Solution Approach 1:
The patent transforms the domain of computation from real domain to logarithmic domain by changing the parameter representation of data. This parameter transformation allows multiplication operations to be converted into addition operations, significantly reducing hardware complexity while maintaining computational accuracy through the mathematical property that log(a×b) = log(a) + log(b).
Solution Approach 2:
The patent replaces the mechanical multiplication operation (which requires complex multiplier circuits) with addition operation in the logarithmic domain. This substitution eliminates the need for multipliers and replaces them with simpler adders, thereby reducing hardware complexity while preserving computational results through logarithmic transformation.
2Productivity
If DNN computation scale is increased to improve performance, then processing capability is enhanced, but amount of computation and memory usage increase
Solution Approach 1:
By changing the computational domain to logarithmic space, the patent reduces the computational complexity of each operation. This parameter transformation enables the system to handle larger scale DNN computations with reduced energy consumption, as addition operations in logarithmic domain require less computational resources than multiplication operations in real domain.
3Adaptability or versatility
If hardware resources are increased to support larger DNN models, then model capacity is improved, but cost and device complexity increase
Solution Approach 1:
The patent applies parameter transformation to convert complex multiplication-based DNN operations into addition-based operations in logarithmic domain. This enables the system to support larger and more versatile DNN models without proportionally increasing hardware resources, as the logarithmic transformation simplifies the fundamental computational operations required.
Data Source
AI summary
Disclosed is a neural network accelerator including a maximum value determiner outputting a maximum value based on a first magnitude component corresponding to first input data and a second magnitude component corresponding to second input data, a sign determiner outputting a sign component corresponding to the maximum value among a first sign component corresponding to the first input data and a second sign component corresponding to the second input data, as an output sign component, an offset operator quantizing a difference between the first magnitude component and the second magnitude component and outputting an output offset based on the first sign component, the second sign component, and the quantization result, and a magnitude operator calculating an output magnitude component of an output data based on the maximum value and the output offset. Each of the first input data and the second input data is data on a logarithm domain.