GPU Tensor Compute Precision Switching for Low-Precision ML
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current graphics processing units (GPUs) face inefficiencies in processing graphics and machine-learning operations due to limitations in parallel processing techniques and precision, particularly in SIMT architectures, leading to suboptimal performance and precision in executing low-precision operations.
Innovation Solution
Implementing a GPU architecture with dynamic precision floating-point units and thread assignment logic to manage and optimize low-precision operations, ensuring efficient execution of machine-learning tasks while minimizing precision loss.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If fixed function computational units are used in GPUs, then processing speed is improved, but precision and versatility for machine-learning operations deteriorate
Solution Approach 1:
The patent implements dynamically configurable computational units that can switch between different precision modes (e.g., 16-bit, 32-bit floating point, integer operations) based on the specific machine-learning task requirements. This dynamic reconfiguration allows the same hardware to optimize for speed in graphics operations while maintaining precision for machine-learning workloads, resolving the contradiction between processing speed and adaptability.
Solution Approach 2:
The computational units are designed with adjustable precision parameters that can be modified at runtime. By changing the precision parameter settings, the GPU can adapt to different operation types - using lower precision for speed-critical graphics rendering and higher precision for accuracy-critical machine-learning computations, thus simultaneously achieving both speed improvement and precision maintenance.
2Productivity
If parallel processing techniques are implemented in SIMT architecture, then productivity is improved, but measurement precision for low-precision operations deteriorates
Solution Approach 1:
The patent segments the parallel processing threads into different precision groups, where threads performing low-precision operations can be handled by specialized low-precision computational units that maintain accuracy, while other threads handle high-precision tasks. This segmentation allows the system to achieve high overall productivity through parallelism while preserving measurement precision for specific low-precision operation groups.
Solution Approach 2:
The patent introduces precision management intermediaries - specialized computational units and control logic that mediate between the parallel processing requirements and precision requirements. These intermediaries ensure that low-precision operations maintain their required precision levels even when executed in parallel, by providing dedicated processing paths and precision-preserving operations for low-precision data types.
Data Source
Figure 1
Figure 2A
Figure 2B
AI summary
In some example embodiments, there may be provided a multi-chip module accelerator usable to execute tensor data processing operations, the multi-chip module accelerator comprising a multi-chip module comprising a memory stack including multiple memory dies; and parallel processor circuitry communicatively coupled to the memory stack. The parallel processor circuitry comprising multiprocessor cores to execute matrix multiplication and accumulate operations, wherein the matrix multiplication and accumulate operations comprise floating-point operations; the floating-point operations are configurable to comprise two-dimensional matrix multiply and accumulate operations involving inputs that have differing floating-point precisions; the floating-point operations comprise a first operation at a first precision and a second operation at a second precision; and the first operation comprises a multiply having at least one 16-bit floating-point input and the second operation comprises an accumulate having a 32-bit floating-point input. Related systems, non-transitory machine-readable storage medium, methods and articles of manufacture are also described.