Neural Network Weight Arrangement for Parallel MAC Simplification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing model training methods lack an explicit rule for determining which weights to prune, leading to inefficient model operation in hardware capable of parallel multiply-accumulate operations, especially when duplicate or small weights appear, limiting performance improvements.
Innovation Solution
A method for optimizing model operation through weight arrangement by obtaining and rearranging weights based on their characteristics, using statistical methods like histograms to identify weights with the same value, opposite signs, or zeros, and applying specific rules to simplify multiply-accumulate operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If model pruning is performed by setting weights to 0 based on weight size, then operation speed is improved, but hardware parallel processing efficiency is not fully utilized due to lack of explicit pruning rules
Solution Approach 1:
The patent applies preliminary action by performing weight arrangement before model operation. Specifically, weights are sorted and arranged in a predetermined order (e.g., ascending or descending order) prior to being used in multiply-accumulate operations. This preliminary arrangement enables hardware to efficiently identify and skip zero or near-zero weights during parallel processing, thereby fully utilizing hardware parallel processing capabilities while maintaining improved operation speed.
Solution Approach 2:
The patent applies parameter changes by transforming the arrangement state of weights from random to ordered. By changing the spatial arrangement parameter of weights (sorting them by magnitude), the system enables hardware to quickly identify redundant computations without changing the fundamental pruning mechanism. This parameter transformation allows explicit pruning rules to be applied, improving both operation speed and hardware utilization efficiency.
2Reliability
If regularization operation is applied during model training, then overfitting is prevented, but many weights become very small and contribute little to the model
Solution Approach 1:
The patent applies the taking out principle by extracting and identifying weights with very small values (near-zero weights) that result from regularization operations. These insignificant weights are then arranged in a specific position (e.g., at the beginning or end of the weight array) through sorting. This extraction and repositioning allows the model operation to skip or ignore these weights, thereby improving operation efficiency while maintaining the reliability benefits of regularization.
Solution Approach 2:
The patent applies discarding and recovering by identifying weights that contribute negligibly to the model output and effectively discarding them through strategic arrangement. By sorting weights and positioning near-zero weights in easily identifiable locations, the system enables the operation to skip these weights without affecting model accuracy. This selective discarding of redundant computational operations recovers computational resources and improves overall model operation efficiency.
3Loss of energy
If duplicate weights or very small weights appear in parallel multiply-accumulate operations, then computation redundancy increases, but no explicit rule exists to determine which weights to prune
Solution Approach 1:
The patent applies parameter changes by transforming the arrangement parameter of weights from random to sorted order. By sorting weights according to their magnitude values, the system creates an explicit rule-based arrangement that makes duplicate and near-zero weights easily identifiable. This simple parameter change (sorting) reduces redundant computation energy by enabling systematic pruning without significantly increasing device complexity, as sorting can be performed once during preprocessing.
Data Source
AI summary
A method for optimizing model operation through weight arrangement and a computing system are provided. The method is operated in an operating device. In the method, a model framework is decided, and a training set is provided according to the model framework for training a model through a learning algorithm. A plurality of weights are computed for the model. The computing system relies on characteristics of the weights to select one of weight-arrangement rules, or a combination of the weight-arrangement rules, so that the locations of all or part of the weights can be re-arranged based on the selected weight-arrangement rule. The re-arranged weights are referred to for designating a corresponding loss function for simplifying the algorithm of the model. An application device can accordingly operate the model.