Arithmetic Device Common Exponent Mantissa Neural Network
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for floating-point number calculations in deep learning, such as those using IEEE-754 format and 16-bit floating-point numbers for neural network training, either fail to reduce calculation complexity or compromise on accuracy.
Innovation Solution
An arithmetic device and method that determine a common exponent and mantissa for multiple floating-point numbers, allowing for efficient four arithmetic operations using 32-bit floating-point numbers with a 1-bit sign, 8-bit exponent, and 23-bit mantissa, specifically optimized for neural network processing layers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If 16-bit floating-point numbers are used for neural network training, then the amount of calculation is reduced, but the calculation accuracy becomes lower than 32-bit precision
Solution Approach 1:
The 32-bit floating-point number is segmented into exponent part and mantissa part. The exponent part is processed separately from the mantissa part, allowing parallel processing and reducing calculation complexity while maintaining full 32-bit precision accuracy.
Solution Approach 2:
The exponent part is extracted and determined once as a common value for multiple calculations. This extracted exponent is then reused across multiple mantissa calculations, reducing redundant operations while preserving complete precision.
2Measurement precision
If standard IEEE-754 floating-point format is used, then calculation accuracy is maintained, but the amount of calculation remains large
Solution Approach 1:
The floating-point calculation is segmented into independent exponent processing and mantissa processing stages. This segmentation allows the exponent to be determined once and reused, eliminating redundant operations while maintaining IEEE-754 compliance and full accuracy.
Solution Approach 2:
The exponent part is determined in advance as a common value before performing multiple mantissa calculations. This preliminary determination of the exponent eliminates redundant exponent processing in subsequent calculations, significantly improving efficiency while maintaining accuracy.
Data Source
AI summary
A processor determines an exponent common to a plurality of numerical values, determines a mantissa for each of the plurality of numerical values based on the determined exponent, and performs four arithmetic operations using a sign, the determined exponent, and the determined mantissa.