Weight Normalization Circuit Using Bit Shifts Instead of Division
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional methods for weight normalization in integrated circuit devices require division operators, which consume significant circuit resources and are costly, especially in applications like digital signal processing and image processing.
Innovation Solution
A method and circuit for weight normalization that performs iterative bit shifts and additions/subtractions to normalize weights without using division operators, utilizing a series of normalization blocks to effectively distribute an expected sum into normalized weights, reducing resource allocation costs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If division operators are used for weight normalization, then normalization accuracy is improved, but circuit resource consumption increases significantly
Solution Approach 1:
The patent transforms the division operation into a multiplication operation by using the reciprocal of the sum of weights. Instead of performing division (which requires complex circuitry), the system calculates the reciprocal of the weight sum and multiplies it with individual weights, achieving the same normalization effect with simpler hardware resources.
Solution Approach 2:
The patent replaces the mechanical division operation with an equivalent mathematical operation (multiplication by reciprocal). This substitution eliminates the need for complex division circuitry while maintaining normalization accuracy, as multiplication can be performed using simpler hardware components.
2Adaptability or versatility
If division operators are used for weight normalization, then normalization function is achieved, but hardware cost increases
Solution Approach 1:
The patent changes the operational parameter from division to multiplication by reciprocal. This parameter change enables the same normalization function to be implemented with lower-cost hardware components, as multiplication units are generally less expensive and more resource-efficient than division units in digital circuit implementation.
Solution Approach 2:
The patent uses a computationally lighter approach (multiplication instead of division) that consumes fewer hardware resources. This allows the system to achieve the normalization function with simpler, more cost-effective circuit elements, reducing overall hardware manufacturing costs.
3Productivity
If parallel division operations are performed, then processing speed is improved, but resource consumption increases exponentially
Solution Approach 1:
The patent changes the operation type from division to multiplication, which enables more efficient parallel processing. Since multiplication operations are fundamentally less resource-intensive than division operations, multiple parallel multiplication units can be instantiated with proportionally fewer resources, allowing faster processing without exponential resource growth.
Data Source
AI summary
Weight normalization in hardware or software without a division operator is described, using only right bit shift, addition and subtraction operations. A right bit shift is performed on an expected sum to effectively divide the expected sum by two to provide a first updated value for the expected sum. An iteration is performed which includes: incrementing with a first adder a first variable by the first updated value of the expected sum to provide an updated value for the first variable; subtracting with a first subtractor a second weight from a first weight to provide a first updated value for the first weight; and performing a left bit shift on the second weight to effectively multiply the second weight by two to provide a first updated value for the second weight.