Custom Floating-Point Format With Non-Integer Base for Neural Training
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing floating-point systems face issues such as precision loss, rounding errors, and suboptimal optimization, particularly in emerging AI applications, leading to inefficient computational performance and increased energy consumption.
Innovation Solution
A custom floating-point number format is introduced, utilizing a non-integer base value between 1 and 2, combined with a sign and exponent field, allowing for improved numerical calculations and database conversions, optimizing the floating-point system for enhanced precision and reduced hardware requirements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional floating-point formats (FP64, FP32, FP16) are used, then versatility in representing numbers is maintained, but computational efficiency is suboptimal and energy consumption increases
Solution Approach 1:
The patent changes the fundamental parameters of the floating-point system by introducing a non-integer base value (between 1 and 2) instead of the traditional integer base (2). This parameter change enables more efficient representation and computation, reducing the number of bits required while maintaining precision, thereby improving computational efficiency and reducing energy consumption
2Productivity
If custom floating-point formats with fewer precision bits are used, then computational unit size is reduced and operations are accelerated, but resolution loss occurs
Solution Approach 1:
The patent introduces a non-integer base value as a key parameter change that allows the system to maintain high resolution with fewer bits. The base value between 1 and 2 enables more efficient encoding of numerical values, preserving precision while reducing the bit width required for representation, thus achieving both faster operations and maintained resolution
3Measurement precision
If the number of mantissa bits is increased to improve precision, then calculation accuracy improves, but hardware multiplier size increases with the square of mantissa bits
Solution Approach 1:
The patent fundamentally changes the base parameter from an integer (traditionally 2) to a non-integer value between 1 and 2. This parameter change transforms the relationship between precision and hardware size, allowing high calculation precision to be achieved with fewer mantissa bits, thereby preventing the quadratic increase in multiplier size that would otherwise occur
Data Source
AI summary
The application discloses a processing method and device for floating-point numbers, a neural network (NN) training method, and a floating-point number database conversion method. A custom floating-point number is acquired, wherein the custom floating-point number includes a sign field and an exponent field, and a numerical value of the custom floating-point number is determined by the sign field, the exponent field, a base value, and a bias value, where the base value is a non-integer value greater than 1 and less than 2. The custom floating-point number is applied in numerical calculations.