Mixed-Precision Compute Array for Linear Algebra
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing processor systems face inefficiencies in computational throughput due to resource-intensive matrix math operations, which are often constrained to fixed precision computations, leading to reduced efficiency and additional rescaling operations when inputs have different precision than results.
Innovation Solution
A computer-implemented method that determines the precision and shape of input matrices and performs parallel linear algebra operations within a compute array to update a result matrix, allowing for mixed-precision computations and efficient matrix operations by using a single accumulator register for both reading and writing data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If fixed precision computations are used in matrix math operations, then computational simplicity is maintained, but computational efficiency and throughput are reduced
Solution Approach 1:
The compute array is designed to dynamically adapt its precision handling capabilities. The system can switch between different precision modes (e.g., single-precision, half-precision, integer operations) based on the specific computational requirements, allowing optimal performance for different workloads without being constrained to a fixed precision mode
Solution Approach 2:
The invention changes the precision parameter of computations performed by the compute array. By supporting multiple precision levels (full precision, half precision, integer operations), the system can optimize computational throughput for different applications, such as using half-precision for machine learning inference where lower precision is acceptable, or full precision for scientific computations requiring higher accuracy
2Measurement precision
If rescaling operations are performed to match precision between inputs and results, then computational accuracy is maintained, but processing time increases
Solution Approach 1:
The system performs precision determination and configuration in advance, before the actual matrix computations begin. By determining the required precision level upfront and configuring the compute array accordingly, the system avoids the need for time-consuming rescaling operations during the computation process, as the appropriate precision mode is already established
Solution Approach 2:
The compute array is designed with universal precision handling capabilities that can process inputs and produce results at multiple precision levels without requiring separate dedicated hardware for each precision level. This multi-functionality eliminates the need for additional rescaling operations, as the same compute array can natively handle different precision requirements
3Reliability
If multiple accumulator registers are used to store intermediate results, then computational accuracy is improved, but resource usage increases
Solution Approach 1:
The invention merges the functionality of multiple accumulator registers into a single accumulator register within each compute element. By combining the accumulation operations and using shared storage resources, the system maintains computational reliability through proper accumulation logic while significantly reducing the quantity of register resources required compared to having separate accumulators for each operation
4Measurement precision
If higher precision operations are performed, then result accuracy is improved, but computational throughput decreases
Solution Approach 1:
The compute array dynamically adjusts its operational precision based on the specific computational task requirements. For applications where high accuracy is critical (e.g., scientific simulations), the system operates in full-precision mode. For applications where approximate results are acceptable (e.g., machine learning inference, graphics processing), the system switches to lower precision modes (half-precision, integer operations), thereby maintaining high computational throughput while achieving sufficient result accuracy
Data Source
AI summary
Aspects include a compute array of a processor with mixed-precision numerical linear algebra support. A first precision and a first shape of a first input matrix and a second precision and a second shape of a second input matrix to the compute array are determined. A plurality of linear algebra operations is repeated in parallel within the compute array to update a result matrix in an accumulator register based on the first input matrix, the second input matrix, and a number of rank updates of the result matrix to store in the accumulator register.