Context-Dependent Deep Learning With Mixed-Format GPU Units
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning algorithms face challenges in efficiently utilizing parallel processing architectures, particularly with low precision fixed point hardware, leading to suboptimal performance and resource allocation in tasks such as deep neural network training and inference.
Innovation Solution
Implementing a graphics processing unit (GPU) with a single instruction, multiple thread (SIMT) architecture to optimize parallel processing for machine learning operations, allowing efficient allocation and execution of tasks across multiple processing clusters.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If low precision fixed point hardware is used for parallel processing, then system cost is reduced, but measurement precision and manufacturing precision deteriorate
Solution Approach 1:
The patent applies different precision levels to different hardware components within the same system. Specifically, it uses a mix of low-precision fixed-point units for cost-effective parallel processing and high-precision floating-point units for critical calculations requiring accuracy. This local differentiation allows the system to optimize overall cost while maintaining necessary precision in specific areas.
Solution Approach 2:
The patent segments the processing tasks into different precision requirements. It divides the computational workload between low-precision fixed-point hardware for bulk parallel operations and high-precision floating-point hardware for operations requiring accuracy. This segmentation allows each component to operate at its optimal precision level, reducing overall system cost while maintaining required precision.
2Productivity
If parallel processing architecture is implemented, then productivity is improved, but device complexity increases
Solution Approach 1:
The patent designs a unified parallel processing architecture where the same hardware structure can handle both low-precision fixed-point operations and high-precision floating-point operations. This multi-functionality reduces device complexity by avoiding the need for completely separate processing paths, while still maintaining high productivity through parallel execution of multiple thread blocks.
Solution Approach 2:
The patent implements dynamic resource allocation in the parallel processing architecture. Thread blocks can be dynamically assigned to different processing units based on their precision requirements. This dynamic approach allows the system to maintain simple hardware structures while achieving high productivity through flexible, context-dependent resource utilization.
3Productivity
If high capacity networks are trained on larger datasets, then productivity is improved, but use of energy increases
Solution Approach 1:
The patent changes the precision parameter of computations during different training phases. It uses low-precision fixed-point arithmetic for the majority of training operations on large datasets, which significantly reduces energy consumption compared to high-precision floating-point arithmetic. Critical operations that require accuracy use higher precision, creating an optimal balance between training capacity and energy usage.
Data Source
Figure 1
Figure 2A
Figure 2B
AI summary
In an example, an apparatus comprises a plurality of execution units comprising and logic, at least partially including hardware logic, to receive a plurality of data inputs for training a neural network, wherein the data inputs comprise training data and weights inputs; represent the data inputs in a first form; and represent the weight inputs in a second form. Other embodiments are also disclosed and claimed.