Fixed-Point to Floating-Point Conversion With Dynamic Range Constraining
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep neural networks face high computational complexity and challenging training processes, particularly when processing tanh and sigm functions in real-time hardware accelerators, which often require expensive hardware and result in accuracy degradation due to approximation and quantization noise.
Innovation Solution
A method and circuit for constraining the dynamic range of data in deep neural networks by converting fixed-point input values to floating-point output values using CLZ/CLO logic, shift blocks, and mask logic, allowing for efficient hardware implementation and reducing the need for full accumulator size, thereby saving power and silicon area.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If fixed-point to floating-point conversion is performed without dynamic range constraining, then conversion accuracy is maintained, but hardware complexity and power consumption increase due to full accumulator size requirements
Solution Approach 1:
The patent applies preliminary action by performing dynamic range constraining before the fixed-point to floating-point conversion. The system determines an additional shifting amount based on the dynamic range of the input data, then applies this shifting to constrain the data within a specific range before conversion. This preliminary constraining allows the use of reduced-precision accumulators while maintaining conversion accuracy, thereby reducing hardware complexity and power consumption.
Solution Approach 2:
The patent utilizes parameter changes by dynamically adjusting the shifting amount based on the input data's dynamic range. The system calculates the appropriate shifting value to constrain the data within a target range, and this shifting parameter is applied during the conversion process. By changing the data representation parameters (shifting amount) based on input characteristics, the system achieves accurate conversion with reduced hardware resources.
2Measurement precision
If full accumulator size is used for fixed-point to floating-point conversion, then conversion precision is maintained, but power consumption and silicon area increase
Solution Approach 1:
The system performs preliminary dynamic range constraining by determining an additional shifting amount before conversion. This preliminary action ensures that the input data is constrained within a range that can be accurately processed by reduced-precision accumulators, thereby maintaining conversion precision while reducing power consumption and silicon area requirements.
Solution Approach 2:
The patent applies parameter changes by dynamically adjusting the shifting amount based on the input data's dynamic range characteristics. This parameter adjustment allows the system to use smaller accumulators with reduced power consumption while maintaining the necessary conversion precision through adaptive data scaling.
3Speed
If approximation methods are used for tanh and sigm functions in hardware accelerators, then computational speed increases, but accuracy degradation occurs
Solution Approach 1:
The patent applies preliminary action by constraining the input data to a specific dynamic range before it is processed by the approximation hardware for tanh and sigm functions. By pre-constraining the data within an optimal range, the approximation methods achieve higher accuracy while maintaining computational speed, as the constrained input reduces the error impact of the approximation.
Solution Approach 2:
The system utilizes parameter changes by dynamically adjusting the data representation (through shifting) to optimize the input range for approximation functions. This parameter adjustment ensures that the approximation methods operate within their most accurate range, thereby improving function approximation accuracy while maintaining the speed benefits of hardware acceleration.
Data Source
AI summary
A method of constraining data represented in a deep neural network is described. The method includes determining an initial shifting specified to convert a fixed-point input value to a floating-point output value. The method also includes determining an additional shifting specified to constrain a dynamic range during converting of the fixed-point input value to the floating-point output value. The method further includes performing both the initial shifting and the additional shifting together to form a dynamic, range constrained, normalized floating-point output value.