16-bit Floating-Point Encoding for Neural Network Hardware Efficiency
Solution Overview
Problem
Existing artificial neural network (ANN) devices face inefficiencies in processing large volumes of input data in real-time, requiring significant computation and inefficient hardware resource utilization due to high-precision operations for low-precision data types.
Innovation Solution
The method involves encoding 16-bit half floating-point input data into 4-bit units by adjusting the exponent and mantissa bits, allowing for efficient splitting and processing, and performing multiply-accumulate operations between 4-bit fixed-point and 16-bit half floating-point data types, optimizing hardware usage by aligning accumulation positions based on exponent differences.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If high-precision operations are used for low-precision data types, then processing accuracy is maintained, but hardware resource utilization becomes inefficient
Solution Approach 1:
The patent changes the parameter representation by converting 16-bit half floating-point numbers into a custom 4-bit unit format with adjusted exponent and mantissa bit allocations. This parameter transformation enables efficient 4-bit unit processing while preserving the essential precision information, resolving the contradiction between maintaining accuracy and improving hardware utilization.
Solution Approach 2:
The patent segments the 16-bit floating-point number into 4-bit units by redistributing the exponent and mantissa bits. Specifically, it assigns 4 bits to the exponent and 11 bits to the mantissa, then further divides the mantissa into multiple 4-bit segments. This segmentation allows parallel processing and efficient hardware resource allocation while maintaining processing accuracy.
2Productivity
If real-time processing of large data volumes is implemented, then processing speed increases, but computational complexity increases
Solution Approach 1:
The patent segments data into 4-bit units, which enables parallel processing and reduces the computational complexity per operation. By processing data in smaller, manageable units, the system can handle large data volumes in real-time without requiring overly complex computational operations.
Solution Approach 2:
The patent uses 4-bit fixed-point arithmetic for MAC operations, which is a simplified partial action compared to full 16-bit floating-point operations. This partial action approach reduces computational complexity while maintaining sufficient precision for neural network operations, enabling real-time processing of large data volumes.
3Productivity
If multiply-accumulate operations are performed with aligned accumulation positions, then processing efficiency improves, but data alignment complexity increases
Solution Approach 1:
The patent performs preliminary alignment of accumulation positions based on exponent differences before executing MAC operations. By pre-calculating and applying alignment offsets, the system simplifies the subsequent accumulation process and improves processing efficiency without requiring complex real-time alignment operations.
Data Source
AI summary
An encoding method includes receiving input data represented by a 16-bit half floating point, adjusting a number of bits of an exponent and a mantissa of the input data to split the input data into 4-bit units, and encoding the input data in which the number of bits has been adjusted such that the exponent is a multiple of “4”.