Combined MX Sparsity Representation for Matrix Accelerator Throughput

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

AI workloads trained in an over-parameterized fashion result in low entropy due to parameter redundancy, leading to inefficient compute throughput.

Innovation Solution

Implement model pruning to increase entropy by removing parameters, thereby enhancing compute throughput.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If model pruning is applied to increase entropy and compute throughput, then compute throughput is improved, but model accuracy may deteriorate

Engineering Contradiction:

Improvecompute throughputVSAvoidmodel accuracy

Core Design Contradiction:

ProductivityVSReliability

Solution Approach 1:

The patent segments the model parameters into different groups based on their importance and redundancy patterns. By dividing the parameter space and applying selective pruning to specific segments, the system achieves entropy increase and compute throughput improvement while preserving critical parameters that maintain model accuracy.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent applies different pruning strategies to different regions of the model based on local characteristics. Important parameters with high contribution to accuracy are protected from pruning, while redundant parameters with low contribution are aggressively pruned. This local differentiation enables simultaneous optimization of throughput and accuracy.

Inventive Principle:

Principle #3Local quality

2Productivity

If parameters are removed to increase entropy, then compute throughput is improved, but model complexity changes

Engineering Contradiction:

Improvecompute throughputVSAvoidmodel complexity

Core Design Contradiction:

ProductivityVSDevice complexity

Solution Approach 1:

The patent implements dynamic model complexity management where the model structure adapts based on operational requirements. The system can adjust the level of pruning and parameter removal dynamically, allowing the model to operate with reduced complexity for higher throughput during inference while maintaining full complexity during training.

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The patent changes the parameter representation format and organization to optimize the balance between complexity and throughput. By transforming parameters into different representations and adjusting their storage and computation characteristics, the system achieves improved throughput without proportionally increasing operational complexity.

Inventive Principle:

Principle #35Parameter changes

Data Source

PatentUS20250292350A1Combined MX and sparsity representation

Publication Date: 2025.09.18 INTEL CORP

AI summary

One embodiment provides a graphics processor comprising a memory interface and a processing cluster array including a plurality of processing resources interconnected via a switched interconnect network, at least one of the plurality of processing resources including a matrix accelerator configured to execute an instruction to perform a multi-dimensional sparse matrix multiply and accumulate operation having input in a sparse microscaling format including merged sparsity and scaling metadata.