Quad Narrowing Operation for Neural Network Precision
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Neural networks face high energy consumption and accuracy loss due to the use of floating-point processing units for floating-point data, which can be mitigated by quantization to fixed-point data but results in reduced accuracy.
Innovation Solution
Implementing a quad narrowing operation that selectively replaces fixed-point operations with floating-point operations in neural networks, using a 32-bit floating-point format for computations and converting outputs to an 8-bit integer format, thereby reducing processing time and maintaining accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If floating-point processing units are used for neural network computations, then computational accuracy is maintained, but energy consumption increases significantly
Solution Approach 1:
The patent applies parameter changes by dynamically adjusting the precision format (floating-point vs. fixed-point) based on computational requirements. The system switches between different data representation parameters to optimize the trade-off between accuracy and energy consumption, using floating-point only when necessary and fixed-point for routine operations.
Solution Approach 2:
The patent implements dynamic operation by allowing the neural network to adaptively switch between floating-point and fixed-point processing modes during computation. This dynamic adjustment enables the system to maintain accuracy when needed while reducing energy consumption during operations where lower precision suffices.
2Use of energy by moving object
If quantization to fixed-point data types is applied, then energy consumption is reduced, but accuracy is lost
Solution Approach 1:
The patent changes the precision parameter dynamically, switching between fixed-point and floating-point representations. This allows the system to use energy-efficient fixed-point arithmetic for operations where full precision is not required, while resorting to floating-point when accuracy requirements demand it.
Solution Approach 2:
The patent segments the neural network computation into different parts that can be processed using different precision levels. Critical path operations or operations requiring high accuracy use floating-point, while less critical operations use fixed-point, thereby reducing overall energy consumption without significantly compromising accuracy.
3Measurement precision
If more operations are performed to maintain accuracy, then computational precision is improved, but processing time increases
Solution Approach 1:
The patent changes operational parameters by selecting different data types and computation modes based on accuracy requirements. This parameter adjustment allows the system to perform fewer, more efficient operations when full precision is not needed, reducing processing time while maintaining sufficient accuracy.
Data Source
AI summary
Systems and methods are disclosed for implementing a quad narrowing operation. The quad narrowing operation converts an output of a 32 bit floating-point operation to the 8 bit integer format by rounding the 32 bit floating-point operation and clamping the rounded 32 bit floating-point input by an 8 bit lower bound and an 8 bit upper bound which are defined in a 16 bit scalar register to generate the fixed-point output. The 8 bit lower bound is defined by the 8 most significant bits of the 16 bit scalar register and the 8 bit upper bound is defined by the 8 least significant bits of the 16 bit scalar register.