Neural Network Weight Sequence Optimization for AI Chip Power Efficiency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Neural networks with large numbers of nodes or neurons require significant computational resources and power due to the computationally exhaustive nature of MAC operations, leading to high energy consumption and inefficiencies in MAC circuitry.
Innovation Solution
A method is introduced to optimize MAC operations by determining a sequence of subsets of weights based on hamming distances, segmenting weight matrices, and rearranging the sequence to minimize bit flips and energy consumption, using a sequence generator to identify and apply the optimal sequence for MAC operations in MAC circuitries.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If MAC operations are performed with standard weight sequences, then neural network computation is completed, but energy consumption is high due to frequent bit flips
Solution Approach 1:
The patent applies preliminary action by pre-ordering weight matrices to minimize Hamming distance before MAC operations are executed. The sequence generator determines an optimal sequence of weight subsets that reduces bit flips in advance, so that when MAC operations proceed, energy consumption is already minimized without sacrificing computational throughput.
Solution Approach 2:
The patent implements dynamics by making the weight sequence adaptive rather than static. The sequence generator dynamically determines the optimal ordering of weight subsets based on their Hamming distances, allowing the system to adapt the computation sequence to minimize energy consumption while maintaining computational efficiency.
2Use of energy by stationary object
If weight matrices are processed in standard sequence, then MAC operations are performed, but number of bit flips is high leading to increased power consumption
Solution Approach 1:
The patent applies segmentation by dividing the weight matrix into multiple weight subsets that can be processed in an optimized sequence. This segmentation allows the sequence generator to independently optimize the ordering of each subset based on Hamming distance metrics, reducing overall bit flips without requiring complete reordering of all weights.
Solution Approach 2:
The patent implements parameter changes by transforming the weight matrix from a standard sequential format into an optimized sequence based on Hamming distance parameters. The sequence generator modifies the ordering parameter of weight subsets to minimize bit transitions, thereby reducing power consumption without changing the fundamental MAC operation structure.
3Loss of energy
If Hamming distance optimization is applied to weight sequences, then energy consumption is reduced, but computation time may increase due to sequence optimization
Solution Approach 1:
The patent applies preliminary action by performing sequence optimization offline before actual MAC operations. The sequence generator determines the optimal weight subset ordering in advance based on Hamming distance calculations, so that during inference or forward propagation, the system simply follows the pre-determined sequence without real-time optimization overhead.
Solution Approach 2:
The patent implements partial action by applying Hamming distance optimization only to critical weight subsets that have the greatest impact on energy consumption, rather than optimizing all weights uniformly. This selective approach reduces computation time while still achieving significant energy savings.
Data Source
AI summary
Disclosed herein includes a system, a method, and a device for improving power efficiency of a neural network implemented in an AI chip. In a neural network, large amounts of computations for multiply and accumulate can result in frequent toggles or transitions in states of logic circuits in the AI chip. Such frequent toggles or transitions of states of logic circuits can cause a large overall power consumption. In one aspect, to minimize the number of toggles, a sequence or order of computations can be rearranged. In one approach, total hamming distances for weights or input strings in different arrangements or sequences can be identified, and an arrangement or a sequence of weights or input strings with a reduced or minimum total hamming distance can be identified. An arrangement or a sequence of weights that render a reduced total hamming distance can be identified.