Matrix Multiply Data Compression Using Exponent-Based Keep Logic

Solution Overview

Problem

Matrix multiplication operations in high performance computing and machine learning are time-consuming and energy-intensive due to the large number of calculations required, necessitating more efficient methods to reduce computational load without compromising accuracy.

Innovation Solution

The implementation of data compression hardware that dynamically determines which product values to drop during matrix multiplication, using keep logic and multiplexor arrays to select the most significant elements for processing, thereby reducing the number of multiply-accumulate operations and energy consumption.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If all product values are processed in matrix multiplication, then calculation accuracy is maintained, but execution time and energy consumption increase

Engineering Contradiction:

Improvecalculation accuracyVSAvoidexecution time

Core Design Contradiction:

Measurement precisionVSLoss of time

Solution Approach 1:

The patent extracts and processes only the most significant product values (those with largest exponent sums) while discarding less significant ones. The keep logic identifies and retains approximately 75% of product values that contribute most to calculation accuracy, eliminating the need to process all product values and thereby reducing execution time while maintaining acceptable accuracy

Inventive Principle:

Principle #2Taking out (Extraction)

Solution Approach 2:

Instead of processing all product values (excessive action), the patent processes only a partial subset (approximately 75%) of the most significant product values. This partial action is sufficient to achieve acceptable calculation accuracy while significantly reducing the computational burden and execution time

Inventive Principle:

Principle #16Partial or excessive action

2Measurement precision

If all product values are processed in matrix multiplication, then calculation accuracy is maintained, but energy consumption increases

Engineering Contradiction:

Improvecalculation accuracyVSAvoidenergy consumption

Core Design Contradiction:

Measurement precisionVSUse of energy by moving object

Solution Approach 1:

The patent extracts and processes only the most significant product values (those with largest exponent sums) while discarding less significant ones. The keep logic identifies and retains approximately 75% of product values that contribute most to calculation accuracy, eliminating the need to process all product values and thereby reducing energy consumption while maintaining acceptable accuracy

Inventive Principle:

Principle #2Taking out (Extraction)

Solution Approach 2:

Instead of processing all product values (excessive action), the patent processes only a partial subset (approximately 75%) of the most significant product values. This partial action is sufficient to achieve acceptable calculation accuracy while significantly reducing the computational burden and energy consumption

Inventive Principle:

Principle #16Partial or excessive action

3Measurement precision

If matrix multiplication is performed without approximation, then accuracy is maintained, but computational load increases

Engineering Contradiction:

ImproveaccuracyVSAvoidcomputational load

Core Design Contradiction:

Measurement precisionVSProductivity

Solution Approach 1:

The patent extracts and processes only the most significant product values (those with largest exponent sums) while discarding less significant ones. The keep logic identifies and retains approximately 75% of product values that contribute most to calculation accuracy, eliminating the need to process all product values and thereby reducing execution time while maintaining acceptable accuracy

Inventive Principle:

Principle #2Taking out (Extraction)

Solution Approach 2:

Instead of processing all product values (excessive action), the patent processes only a partial subset (approximately 75%) of the most significant product values. This partial action is sufficient to achieve acceptable calculation accuracy while significantly reducing the computational burden and execution time

Inventive Principle:

Principle #16Partial or excessive action

Data Source

PatentUS12072952B2Data compressor for approximation of matrices for matrix multiply operations

Publication Date: 2024.08.27 ADVANCED MICRO DEVICES INC

AI summary

A processing device is provided which comprises memory configured to store data and a processor. The processor comprises a plurality of MACs configured to perform matrix multiplication of elements of a first matrix and elements of a second matrix. The processor also comprises a plurality of logic devices configured to sum values of bits of product exponents values of the elements of the first matrix and second matrix and determine keep bit values for product exponents values to be kept for matrix multiplication. The processor also comprises a plurality of multiplexor arrays each configured to receive bits of the elements of the first matrix and the second matrix and the keep bit values and provide data for selecting which elements of the first matrix and the second matrix values are provided to the MACs for matrix multiplication.