Neural Network Classification With Compressed Labels and Mixed Precision
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Training and inferencing with neural networks require significant memory, time, and computing resources, which can be improved.
Innovation Solution
The use of compressed representations of class labels and mixed precision training techniques to reduce memory footprint and increase throughput in neural network processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If traditional training and inferencing methods are used with neural networks, then processing capability is maintained, but memory consumption and computing resource usage increase significantly
Solution Approach 1:
The patent applies parameter changes by transitioning from standard floating-point precision to half-precision floating-point numbers (FP16) for neural network weights and activations. This parameter change in numerical precision reduces memory consumption by 4x compared to FP32, while maintaining adequate processing capability through mixed-precision computation that selectively uses higher precision only where necessary for numerical stability.
Solution Approach 2:
The patent segments the computation process into different precision levels, using half-precision for most operations and full precision only when needed. This segmentation allows the system to optimize memory usage by storing weights and activations in compressed FP16 format while performing critical computations in FP32, thereby reducing overall memory footprint without significantly compromising processing capability.
2Measurement precision
If traditional training and inferencing methods are used with neural networks, then processing accuracy is maintained, but computing resource usage increases significantly
Solution Approach 1:
The patent changes the precision parameter from FP32 to FP16 for neural network computations. This parameter change reduces computing resource usage by decreasing the number of operations and memory bandwidth required, while lossless or near-lossless compression techniques and selective full-precision computation preserve processing accuracy for critical operations.
Solution Approach 2:
The patent applies partial precision computation by using half-precision floating-point numbers for most neural network operations where full precision is not strictly necessary, while resorting to full precision only when numerical stability requires it. This partial application of high precision reduces overall computing resource consumption while maintaining adequate accuracy for the majority of computations.
3Quantity of substance
If compressed representations of class labels are used, then memory footprint is reduced, but processing complexity increases
Solution Approach 1:
The patent changes the representation parameter of class labels from traditional one-hot encoding to compressed integer or binary encodings. This parameter change reduces memory footprint by storing class labels in fewer bits (e.g., log2(N) bits for N classes instead of N bits for one-hot encoding), while the increased processing complexity is mitigated by efficient decoding algorithms and hardware-optimized lookup tables that rapidly translate compressed codes back to class identifiers.
Data Source
AI summary
Apparatuses, systems, and techniques to train neural networks to perform classification. In at least one embodiment, one or more neural networks are trained to perform classification based on, for example, using one or more compressed representations of one or more class labels, where the one or more compressed representations have fewer bits than a representation of the one or more class labels.