Neural Network Accelerator Weight Reordering for Lower MAC Switching

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Deep neural networks (DNNs) face significant dynamic power consumption due to high switching activity in MAC units, which current optimization techniques like pruning, quantization, and hardware gating fail to effectively address, particularly in power-constrained environments.

Innovation Solution

Implement a compiler that reorders weights to minimize switching activity by using a pivot-based greedy heuristic sorting algorithm, optimizing weight arrangement to reduce bit transitions and power consumption without altering hardware or accuracy.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Use of energy by moving object

If weights are processed in conventional order in MAC units, then computation accuracy is maintained, but dynamic power consumption increases due to high switching activity

Engineering Contradiction:

Improvedynamic power consumptionVSAvoidswitching activity

Core Design Contradiction:

Use of energy by moving objectVSLoss of energy

Solution Approach 1:

The patent applies preliminary action by reordering weights before they are loaded into the MAC units. The compiler analyzes the weight matrix and rearranges weights to minimize switching activity before execution, so that when the MAC units process the weights, the switching activity is already optimized. This pre-processing step reduces dynamic power consumption without requiring runtime adjustments.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The patent changes the ordering parameter of weights to optimize power consumption. By transforming the weight arrangement from conventional order to an optimized order based on switching activity metrics, the system achieves lower dynamic power consumption while maintaining the same computational functionality.

Inventive Principle:

Principle #35Parameter changes

2Use of energy by moving object

If pruning, quantization, or hardware gating is applied to reduce power consumption, then energy efficiency improves, but computation accuracy or model performance deteriorates

Engineering Contradiction:

Improveenergy efficiencyVSAvoidcomputation accuracy

Core Design Contradiction:

Use of energy by moving objectVSMeasurement precision

Solution Approach 1:

Inventive Principle:

Principle #35Parameter changes

Solution Approach 2:

The patent replaces physical hardware modifications (such as pruning connections, reducing precision, or gating hardware) with a software-based weight reordering strategy. This substitution allows power optimization without altering the fundamental computation structure, thereby preserving accuracy.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

3Use of energy by moving object

If weight reordering is applied to reduce switching activity, then dynamic power consumption decreases, but compiler complexity increases

Engineering Contradiction:

Improvedynamic power consumptionVSAvoidcompiler complexity

Core Design Contradiction:

Use of energy by moving objectVSDevice complexity

Solution Approach 1:

The patent applies preliminary action by performing weight reordering during the compilation phase rather than at runtime. The compiler analyzes the weight matrix and determines the optimal ordering before the model is deployed, so that the actual execution on the accelerator does not require complex real-time reordering logic.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The patent makes the compiler self-service the power optimization task by automatically analyzing the weight matrix and determining the optimal ordering without requiring manual intervention or complex external tools. The compiler itself performs the optimization as part of its standard compilation process.

Inventive Principle:

Principle #25Self-service

4Loss of energy

If weights are reordered to minimize bit transitions, then switching power consumption is reduced, but data access pattern complexity increases

Engineering Contradiction:

Improveswitching powerVSAvoiddata access pattern

Core Design Contradiction:

Loss of energyVSEase of operation

Solution Approach 1:

The patent applies preliminary action by determining the optimal weight ordering during compilation and generating the corresponding access patterns in advance. The reordered weights and their access patterns are both prepared beforehand, so that during execution, the hardware simply follows the pre-determined access pattern without needing to dynamically determine optimal ordering.

Inventive Principle:

Principle #10Preliminary action

Data Source

PatentUS20260004151A1Reducing power consumption of neural network accelerator through weight reordering

Publication Date: 2026.01.01 INTEL CORP

AI summary

To address the switching issue directly, an improved compiler can be implemented to prepare a DNN for hardware execution on a DNN accelerator in a way that considers switching activity during multiply accumulate (MAC) operations and reorders the weights to reduce the switching activity. The resulting compiled DNN can reduce energy consumption through reducing or minimizing switching activity in the DNN and can effectively reduce dynamic power consumption in DNN accelerators. The compiler can determine and enforce an improved weight ordering during the model compilation process. The compiler can be guided by one or more weight arrangement and reordering rules to ensure weights are ordered to minimize or reduce switching activity as much as possible without changing the output accuracy of the model or violating strict data paths of the MAC array. Compiled models with weight reordering would exhibit remarkably lower dynamic power consumption when deployed on DNN accelerators.