Binarized Neural Network Circuit Eliminates Bias Term
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional CNNs require a bias term for accurate weight distribution adjustment, leading to increased area and power consumption due to the need for additional circuits and high bit accuracy, especially as weights become more evenly distributed during learning.
Innovation Solution
A neural network circuit device that includes a logic circuit for binary input and weight processing, a sum circuit for summing output results, a batch normalization circuit to correct variance through normalization and shifting, and an activation circuit using the sgn function, eliminating the need for a bias term by incorporating a scaling coefficient and shift value.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a bias term is used for accurate weight distribution adjustment in CNN, then recognition accuracy is improved, but area and power consumption increase due to additional circuits and high bit accuracy requirements
Solution Approach 1:
The patent transforms the bias term from a high-precision floating-point value into a low-precision integer offset applied after batch normalization. By changing the parameter representation and applying it at a different stage (post-normalization), the circuit can use simple integer addition instead of high-precision floating-point addition, significantly reducing area and power consumption while maintaining recognition accuracy.
2Measurement precision
If a bias term is used for accurate weight distribution adjustment in CNN, then recognition accuracy is improved, but power consumption increases due to additional circuits and high bit accuracy requirements
Solution Approach 1:
The patent changes the bias term from a high-precision floating-point parameter to a low-precision integer offset. This parameter transformation allows the use of simple integer addition circuits instead of high-precision floating-point adders, dramatically reducing power consumption while preserving the functional effect of weight distribution adjustment.
Solution Approach 2:
The patent extracts the bias functionality from the traditional pre-activation position and relocates it to post-normalization. By separating the bias application from the main computational path and applying it after batch normalization, the circuit avoids the need for high-precision bias storage and computation, reducing power consumption.
3Area of stationary object
If binarized accuracy is used to reduce hardware complexity, then area and power consumption are reduced, but recognition accuracy deteriorates
Solution Approach 1:
The patent applies batch normalization to transform the binarized weights and activations into a distribution with zero mean and unit variance. This parameter transformation allows the use of simple binary multiplication (XNOR operations) while maintaining the statistical properties needed for accurate recognition, effectively decoupling hardware simplicity from recognition performance.
Data Source
AI summary
Provided are a neural network circuit device, a neural network, a neural network processing method, and a neural network execution program, each of which does not require a bias. A binarized neural network circuit includes: an input part configured to allow input of an input node which allows input of input values x1-xn (xi) (binary), and input of weights w1-wn (wi) (binary); an XNOR gate circuit configured to receive the input values x1-xn and the weights w1-wn and to take XNOR logic; a sum circuit configured to sum XNOR logical values; a batch normalization circuit configured to correct a variance due to binarization, by extending a range of normalization and shifting a center thereof; and an activating function circuit configured to convert a signal B obtained by batch-normalizing a signal Y generated by taking the sum, by means of an activating function f sgn(B).