16-bit Floating-Point Encoding for Neural Network Hardware Efficiency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing artificial neural network (ANN) devices face inefficiencies in processing large volumes of input data in real-time, requiring significant computation and inefficient hardware resource utilization due to high-precision operations for low-precision data types.
Innovation Solution
The method involves encoding 16-bit half floating-point input data into 4-bit units by adjusting the exponent and mantissa bits, allowing for efficient splitting and processing, and performing multiply-accumulate operations between 4-bit fixed-point and 16-bit half floating-point data types, optimizing hardware usage by aligning accumulation positions based on exponent differences.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If high-precision operations are used for low-precision data types, then processing accuracy is maintained, but hardware resource utilization becomes inefficient
Solution Approach 1:
The patent changes the parameter representation by converting 16-bit half floating-point numbers into a custom 4-bit unit format with adjusted exponent and mantissa bit allocations. This parameter transformation enables efficient 4-bit unit processing while preserving the essential precision information, resolving the contradiction between maintaining accuracy and improving hardware utilization.
Solution Approach 2:
The patent segments the 16-bit floating-point number into 4-bit units by redistributing the exponent and mantissa bits. Specifically, it assigns 4 bits to the exponent and 11 bits to the mantissa, then further divides the mantissa into multiple 4-bit segments. This segmentation allows parallel processing and efficient hardware resource allocation while maintaining processing accuracy.
2Productivity
If real-time processing of large data volumes is implemented, then processing speed increases, but computational complexity increases
Solution Approach 1:
The patent segments data into 4-bit units, which enables parallel processing and reduces the computational complexity per operation. By processing data in smaller, manageable units, the system can handle large data volumes in real-time without requiring overly complex computational operations.
Solution Approach 2:
The patent uses 4-bit fixed-point arithmetic for MAC operations, which is a simplified partial action compared to full 16-bit floating-point operations. This partial action approach reduces computational complexity while maintaining sufficient precision for neural network operations, enabling real-time processing of large data volumes.
3Productivity
If multiply-accumulate operations are performed with aligned accumulation positions, then processing efficiency improves, but data alignment complexity increases
Solution Approach 1:
The patent performs preliminary alignment of accumulation positions based on exponent differences before executing MAC operations. By pre-calculating and applying alignment offsets, the system simplifies the subsequent accumulation process and improves processing efficiency without requiring complex real-time alignment operations.
Data Source
AI summary
An encoding method includes receiving input data represented by a 16-bit half floating point, adjusting a number of bits of an exponent and a mantissa of the input data to split the input data into 4-bit units, and encoding the input data in which the number of bits has been adjusted such that the exponent is a multiple of “4”.