ML Weight Scale Shifting for Low-Precision Accuracy
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current approaches to using lower-precision data representation formats in machine learning and artificial intelligence models, such as BF16, often result in accuracy loss and require extensive software testing, complicating the deployment process and increasing computational costs, especially in applications requiring high accuracy like finance, robotics, and radiology.
Innovation Solution
Implement a method that scales values from high-precision formats like FP32 to lower-precision formats like BF16 using a weighting factor, and then reverses the scaling post-operation to maintain accuracy, reducing memory footprint and computational demand.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If lower-precision data representation formats (e.g., BF16) are used in machine learning operations, then memory footprint and computational demand are reduced, but accuracy loss occurs
Solution Approach 1:
The patent applies parameter changes by introducing a scale factor that transforms the precision parameter of data representation. By scaling BF16 values by a carefully selected scale factor (e.g., 2^16), the system changes the effective precision parameter to maintain accuracy while using lower-precision storage, thus resolving the contradiction between memory footprint and accuracy.
Solution Approach 2:
The scale factor acts as an intermediary element between high-precision (FP32) and low-precision (BF16) data representations. It mediates the conversion process by allowing values to be stored in BF16 format while maintaining FP32-level accuracy through the scaling relationship, thus enabling reduced memory footprint without sacrificing accuracy.
2Use of energy by moving object
If lower-precision data representation formats are used, then computational cost and resource expenditure are reduced, but accuracy loss occurs
Solution Approach 1:
The patent changes the precision parameter through scaling, allowing computational operations to proceed in BF16 format with reduced computational cost while maintaining FP32-level accuracy. The scale factor ensures that rounding errors in BF16 operations do not accumulate to degrade final results, thus resolving the contradiction between computational cost and accuracy.
3Measurement precision
If extensive software testing is performed to ensure accuracy with lower-precision formats, then accuracy can be maintained, but deployment complexity increases
Solution Approach 1:
The scale factor mechanism provides self-service by automatically ensuring accuracy through mathematical transformation rather than requiring extensive manual testing. The scaling relationship is deterministic and can be verified through simple properties, eliminating the need for complex testing procedures and reducing deployment complexity while maintaining accuracy.
4Measurement precision
If truncation errors are minimized through scaling, then accuracy is maintained, but additional computational operations are required
Solution Approach 1:
The patent applies preliminary action by performing the scaling operation before the main computational work. By pre-scaling BF16 values by the scale factor (e.g., 2^16) before dot product operations, the system minimizes truncation errors during computation. The reverse scaling is performed only after the computation, reducing the overhead of precision maintenance operations while maintaining accuracy.
Data Source
AI summary
Disclosed is a technical solution for improving accuracy of operations of machine learning by compensating for lower precision with scale shifting. An example non-transitory computer readable medium comprises instructions that, when executed, cause a machine to at least identify a first precision data type and a second precision data type associated with execution of a machine-learning model, the first precision data type to have a first data precision greater than a second data precision of the second precision data type, determine at least one scale factor to be applied to first weights of the machine-learning model, the first weights based on the first precision data type, and convert the first weights to second weights based on a multiplication of the first weights and the at least one scale factor, the second weights based on the second precision data type.