Adaptive Floating-Point Containers for Memory-Efficient ML Training
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Training neural networks is an exascale task that requires significant energy and time due to memory transfers, especially in edge computing, where existing techniques like quantization and distributed training do not efficiently adjust data representations during training.
Innovation Solution
Adapt mantissa and exponent bitlengths dynamically during training using machine learning techniques like Quantum Mantissa and Quantum Exponent to reduce memory footprint and energy consumption, employing lossless and lossy compression methods to optimize floating-point data storage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If standard floating-point data storage is used during neural network training, then training accuracy is maintained, but memory footprint and energy consumption increase significantly
Solution Approach 1:
The patent applies dynamic adaptation of floating-point container bitlengths during training. The system dynamically adjusts the number of bits used to store mantissas and exponents based on training progress and data distribution characteristics. This allows the memory footprint to be reduced from standard full precision to optimized variable precision formats, achieving up to 92% reduction in memory footprint while maintaining training accuracy through continuous adaptation of precision levels.
Solution Approach 2:
The patent changes the parameter of floating-point container bitlengths during training. Specifically, it adapts the mantissa bitlength and exponent bitlength based on the training dataset characteristics and model progress. By modifying these parameters dynamically, the system achieves efficient memory usage while preserving the necessary precision for accurate training, resolving the contradiction between memory footprint and training accuracy.
2Reliability
If full precision floating-point data is stored throughout training, then training accuracy is maintained, but energy efficiency deteriorates due to increased memory transfers
Solution Approach 1:
The system dynamically adapts the precision of floating-point containers during training based on actual training needs. By reducing bitlengths when full precision is not required, the system decreases memory transfer volume and associated energy consumption while maintaining sufficient accuracy for training convergence, achieving 5.17× improvement in energy efficiency.
Solution Approach 2:
The patent changes the precision parameters (mantissa and exponent bitlengths) of floating-point data during training. This parameter adaptation allows the system to use minimal necessary bits for storage and transmission, reducing memory bandwidth requirements and energy consumption while preserving training accuracy through intelligent precision management.
3Ease of manufacture
If fixed bitlengths are used for floating-point data, then data storage is simple, but adaptability to different training stages and datasets is reduced
Solution Approach 1:
The patent transitions from fixed bitlength storage to dynamic adaptive storage. The system automatically adjusts container bitlengths based on training stage, dataset characteristics, and model progress. This dynamic approach maintains ease of implementation through automated adaptation while significantly improving adaptability to different training scenarios, enabling optimal resource utilization across various training phases.
Data Source
AI summary
Provided is a system and method for adaptation of containers for floating-point data for training of a machine learning model. The method includes: receiving training data for training the machine learning model; determining adapted mantissa bitlengths for the floating point data used to store activations or weights of the training data, the adapted mantissa bitlengths are determined independent of the bitlengths of the exponents of the floating point data; and storing the floating point data with the adapted mantissas for training the machine learning model.