Neural Processor Encoding with Fixed Exponents and Variable Mantissas
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing deep learning quantization methods using floating point representation face slow computation speeds due to the need for separate arithmetic units, while fixed point representation lacks flexibility in expressing a wide range of numbers.
Innovation Solution
A neural processor apparatus and method that converts parameter values to a format with a fixed-length exponent and variable-length mantissa using a mapping table based on parameter distribution, enabling efficient computation through a shifter and comparator.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If floating point representation is used, then a larger range of numbers can be expressed, but computation speed decreases requiring separate floating point arithmetic units
Solution Approach 1:
The patent segments the floating-point number representation into two distinct parts: a fixed-length exponent field and a variable-length mantissa field. This segmentation allows the exponent to remain fixed for faster processing while the mantissa can vary in length to represent different ranges, thus maintaining versatility without sacrificing computation speed.
Solution Approach 2:
The patent changes the parameter structure by fixing the exponent length while allowing the mantissa length to vary. This parameter change enables the system to express a large range of numbers (through variable mantissa) while maintaining computation speed (through fixed exponent that can be processed efficiently by standard arithmetic units).
2Productivity
If fixed point representation is used, then computation speed improves, but flexibility in expressing a wide range of numbers decreases
Solution Approach 1:
The patent introduces asymmetry into the number representation by making the mantissa length variable while keeping the exponent length fixed. This asymmetric structure provides flexibility (variable mantissa can adapt to different precision needs) while maintaining the computational efficiency of fixed-point operations (fixed exponent).
Solution Approach 2:
The patent makes the mantissa length dynamic rather than fixed. The variable-length mantissa can be adjusted based on the specific computational requirements, providing flexibility while the fixed exponent maintains computation speed. This dynamic structure allows the system to adapt between precision and range needs.
3Quantity of substance
If variable-length mantissa is used, then memory usage is reduced, but complexity of number format increases
Solution Approach 1:
The patent extracts the length information of the mantissa as a separate metadata element. By storing the mantissa length separately from the actual mantissa data, the system can efficiently represent variable-length mantissas without increasing the core computational complexity. This extraction allows memory optimization while keeping the number format manageable.
Data Source
AI summary
Disclosed are a method of expressing a parameter variably and an apparatus for the same. A neural processor apparatus includes: a comparator configured to read a value of a fixed-length exponent of a previously-converted parameter value and obtain mantissa-length information of a mantissa, wherein the mantissa-length information is obtained from a mapping table based on being mapped to the value of the exponent; a shifter configured to read the mantissa of the previously-converted parameter value and use the mantissa-length information to convert a structure of the previously-converted parameter value; and the mapping table, in which the mantissa-length information of the mantissa is mapped to the value of the exponent.