Mixed-Precision NPU Spatial Fusion Load Balancing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing mixed-precision neural processing units (NPUs) face inefficiencies due to desynchronization between computation results from different data/weight subdivisions, leading to stalling and reduced performance, especially when dealing with varying sparsity distributions across precision levels.
Innovation Solution
The implementation of a mixed-precision NPU using spatial fusion with load balancing techniques, where data/weight combinations are alternately loaded across computation circuits to minimize stalling and synchronize partial computation results, leveraging sparsity to optimize computation time and reduce desynchronization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If data/weight subdivisions are processed in parallel across multiple computation circuits, then computation speed is improved, but desynchronization between partial computation results causes stalling and reduces overall productivity
Solution Approach 1:
The system implements periodic action by alternating between different data/weight subdivision combinations in a systematic sequence. Computation circuits process different subdivisions in alternating time slots, allowing results to be generated in a synchronized periodic pattern that eliminates stalling while maintaining high computation speed
Solution Approach 2:
The system applies dynamics by dynamically assigning different data/weight subdivision combinations to computation circuits based on current processing state. The assignment is not static but adapts over time, allowing the system to optimize synchronization and prevent stalling while maintaining parallel processing efficiency
2Manufacturing precision
If high precision computation is used, then manufacturing precision of computation results is improved, but processing time increases and productivity decreases
Solution Approach 1:
The system segments computation into multiple precision levels by dividing data and weight subdivisions into different precision groups. Each computation circuit processes specific precision segments in parallel, allowing high precision results to be generated without requiring all circuits to operate at maximum precision, thus maintaining both accuracy and throughput
Solution Approach 2:
The system changes the precision parameter dynamically across different computation circuits and time slots. By varying the precision level assigned to different data/weight subdivisions, the system optimizes the balance between computation result accuracy and processing speed, achieving high precision where needed without sacrificing overall productivity
3Productivity
If load balancing is implemented to distribute computation evenly, then productivity is improved, but device complexity increases due to additional control circuits
Solution Approach 1:
The load balancing is achieved through periodic action by systematically alternating between different data/weight subdivision combinations in a predetermined sequence. This systematic alternation provides automatic load distribution without requiring complex real-time monitoring or control logic, balancing productivity improvement with acceptable device complexity
Solution Approach 2:
The system implements self-service load balancing where the computation circuits themselves participate in the load distribution through the alternating processing pattern. The inherent structure of processing different subdivisions in alternating sequences automatically balances the load across circuits without requiring external control mechanisms, maintaining simplicity while improving productivity
Data Source
AI summary
According to one general aspect, an apparatus may include a machine learning system. The machine learning system may include a precision determination circuit configured to: determine a precision level of data, and divide the data into a data subdivision. The machine learning system may exploit sparsity during the computation of each subdivision. The machine learning system may include a load balancing circuit configured to select a load balancing technique, wherein the load balancing technique includes alternately loading the computation circuit with at least a first data/weight subdivision combination and a second data/weight subdivision combination. The load balancing circuit may be configured to load a computation circuit with a selected data subdivision and a selected weight subdivision based, at least in part, upon the load balancing technique. The machine learning system may include a computation circuit configured to compute a partial computation result based, at least in part, upon the selected data subdivision and the weight subdivision.