Neural Network Circuit Using Bit Shifting to Reduce Multiplier Count
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep neural networks require a large number of multipliers for circuit implementation, leading to an increase in circuit scale when using logic circuits like FPGAs for neuron operations.
Innovation Solution
A neural network circuit is configured using a combination of multiplication, bit shifting, and addition operations to simplify the multiplication of input data by weight data, employing a method that rounds the mantissa part of weight data and shifts the results according to the exponent part, allowing for a small-scale circuit implementation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a large number of multipliers are used to implement neuron operations in deep neural networks, then the computation accuracy is improved, but the circuit scale increases significantly
Solution Approach 1:
The patent changes the parameter representation of weight data from binary format to a format based on powers of two (2^k). By expressing weights as coefficients multiplied by powers of two, the multiplication operation is transformed into a combination of addition and bit-shifting operations, thereby reducing circuit complexity while maintaining computational accuracy
Solution Approach 2:
The patent replaces the mechanical multiplication operation with a substitution of operations: instead of using multipliers, it uses adders combined with bit-shifting circuits. This substitution leverages the property that multiplying by 2^k is equivalent to left-shifting by k bits, thereby simplifying the hardware implementation
2Device complexity
If weight data is approximated by one power of two or sum of powers of two, then the circuit scale is reduced, but the multiplication accuracy is degraded
Solution Approach 1:
The patent applies partial action by representing weight data as a sum of a limited number of power-of-two terms rather than requiring full precision binary representation. This partial representation reduces the number of terms needed in the summation, thereby simplifying the circuit while maintaining sufficient accuracy for neural network computations
3Device complexity
If bit shifting and addition operations are used instead of multiplication, then the number of multipliers is reduced, but the number of addition operations increases
Solution Approach 1:
The patent segments the weight data into multiple power-of-two components (2^k1 + 2^k2 + ... + 2^kn). Each segment corresponds to a term in the summation, allowing the multiplication to be decomposed into separate addition and bit-shifting operations for each segment, thereby eliminating the need for multipliers
Data Source
AI summary
The present invention addresses the problem of implementing a neural network using a small-scale circuit by simplifying the multiplication of the input data by weight data. The neural network circuit according to the present invention is configured from: a means for multiplying input data by a rounded value of the mantissa part of weight data; a means for shifting the multiplication result by the number of bits of the rounded value; a means for adding the shifted result to the original input data; and a means for shifting the addition result by the number of bits of the exponent part of the weight.