Vector Operator Conversion for Matrix Unit Throughput
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The computational power of vector computation units in intelligent chips is weak, limiting the overall performance of the chip, particularly in neural network computations where vector-type operations with low complexity become a bottleneck.
Innovation Solution
Convert vector operators into matrix multiplication operators when certain conditions are met, utilizing the more powerful matrix computation units to perform these operations, thereby optimizing processing efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Power
If vector operators are executed by vector computation units, then the operations can be performed with low computational complexity, but the weak computational power of vector computation units restricts the overall computational power of the intelligent chip
Solution Approach 1:
The patent merges the execution of vector operators into the matrix computation unit by converting vector operators into equivalent matrix multiplication operators. This allows the matrix computation unit to handle both matrix multiplication and vector operations, effectively combining the functional capabilities of both computation units and eliminating the bottleneck caused by the vector computation unit's limited power.
Solution Approach 2:
The matrix computation unit is designed to perform multiple functions: its original matrix multiplication operations and the converted vector operations. By making the matrix computation unit universal, the system eliminates the need for the vector computation unit to handle certain operations, thereby overcoming the limitation of weak vector computational power while maintaining the ability to perform low-complexity vector operations.
2Power
If the matrix computation unit is designed with high computational power for matrix multiplication, then matrix operations are accelerated, but vector-type operations with low computing complexity become a bottleneck that restricts the matrix computation unit's computational power
Solution Approach 1:
The patent combines the handling of both matrix multiplication and vector operations into a single matrix computation unit. By converting vector operators into matrix multiplication operators, the system allows the high-power matrix computation unit to process vector operations without requiring separate vector computation resources, thereby eliminating the time loss bottleneck.
3Productivity
If vector operators are converted into matrix multiplication operators, then vector-type operations can be accelerated using the matrix computation unit, but additional conversion overhead is introduced
Solution Approach 1:
The patent changes the representation parameters of vector operators by converting them into matrix multiplication operators with specific structures. The conversion follows defined rules where vector operators are transformed into matrices with particular patterns (such as sparse matrices or matrices with specific row/column structures), allowing the matrix computation unit to execute them efficiently while managing conversion complexity through systematic parameter transformation.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A data processing method and a related device are disclosed, to improve utilization of a matrix computation unit in an intelligent chip, improve vector-type operation efficiency, and further improve a throughput of the intelligent chip. The method includes: after a vector operator is obtained, determining whether the vector operator is capable of being converted into an equivalent matrix multiplication operator; and converting the vector operator into a corresponding target matrix multiplication operator if the vector operator is capable of being converted into the equivalent matrix multiplication operator, so that the matrix computation unit in the intelligent chip can execute the target matrix multiplication operator, thereby improving overall operation efficiency of the intelligent chip.