Matrix Processor Floating-Point Operations
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing processor-based systems face inefficiencies in performing floating-point matrix multiplication operations, which are power-intensive and occupy significant chip area, hindering the performance of deep learning applications that rely heavily on such operations.
Innovation Solution
The implementation of a matrix-processor-based device with positive and negative partial sum accumulators that determine the sign of intermediate products during floating-point operations, normalizing and accumulating them efficiently to reduce the number of renormalization cycles, thereby minimizing power consumption and improving processor performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If floating-point matrix multiplication operations are performed using conventional processors, then computational accuracy is maintained, but power consumption increases and chip area is significantly occupied
Solution Approach 1:
The patent segments the accumulation process into separate positive and negative partial sum accumulators. This segmentation allows independent handling of positive and negative values, reducing the frequency of renormalization operations and thereby reducing power consumption while maintaining computational accuracy through precise tracking of signed partial sums.
Solution Approach 2:
The patent changes the parameter of renormalization frequency by implementing a deferred renormalization strategy. Instead of renormalizing after each addition, the system accumulates partial sums and performs renormalization less frequently, changing the temporal parameter of the renormalization operation to reduce overall power consumption.
2Measurement precision
If floating-point matrix multiplication operations are performed using conventional processors, then computational accuracy is maintained, but processor performance is reduced
Solution Approach 1:
The patent applies preliminary action by pre-separating intermediate products into positive and negative categories before accumulation. This preliminary classification enables more efficient accumulation operations and reduces the need for frequent renormalization, thereby improving processor performance while maintaining accuracy through systematic handling of signed values.
Solution Approach 2:
By segmenting the accumulation into positive and negative partial sums, the patent enables parallel processing paths and reduces critical path delays. This segmentation improves processor performance by allowing independent accumulation operations that can be executed more efficiently than conventional sequential floating-point accumulation.
3Measurement precision
If renormalization is performed after each intermediate product addition, then computational accuracy is maintained, but the number of processor cycles increases
Solution Approach 1:
The patent changes the frequency parameter of renormalization operations by deferring them until accumulation is complete. Instead of performing renormalization after each addition (high frequency), the system accumulates multiple partial sums and performs renormalization less frequently, thereby reducing the number of processor cycles required while maintaining accuracy through the deferred renormalization strategy.
Solution Approach 2:
The patent enables continuous accumulation of partial sums without interruption for renormalization. By maintaining continuous accumulation in the positive and negative partial sum accumulators and postponing renormalization, the system reduces the number of processor cycles lost to renormalization operations while preserving computational accuracy.
Data Source
Figure 1A
Figure 1B
Figure 2
AI summary
Providing efficient floating-point operations using matrix processors in processor-based systems is disclosed. In this regard, a matrix-processor-based device provides a matrix processor comprising a positive partial sum accumulator and a negative partial sum accumulator. As the matrix processor processes pairs of floating-point operands, the matrix processor calculates an intermediate product based on a first floating-point operand and a second floating-point operand and determines a sign of the intermediate product. Based on the sign, the matrix processor normalizes the intermediate product with a partial sum fraction of the positive partial sum accumulator or the negative partial sum accumulator, then adds the intermediate product to the positive sum accumulator or the negative sum accumulator. After processing all pairs of floating-point operands, the matrix processor subtracts the negative partial sum accumulator from the positive partial sum accumulator to generate a final sum, then renormalizes the final sum a single time.