Sparse Weight Tensor MAC Operations for AI Chip Efficiency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional data processing devices face challenges in efficiently handling multiply-accumulate (MAC) operations due to increasing computational demands and growing area and power consumption, particularly in neural-network processing units (NPUs) as AI technology advances.
Innovation Solution
A data processing device and method that utilize a sparse weight tensor obtained by sparse processing of the original weight tensor, reducing data points and thereby reducing the computational workload, while maintaining computational performance through a multiplying accumulator that performs MAC operations using the sparse tensor and input tensor.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional MAC operations are implemented using full weight tensors, then computational accuracy is maintained, but computational workload, area consumption, and power consumption increase
Solution Approach 1:
The patent extracts and removes redundant or less important weight elements from the original weight tensor to create a sparse weight tensor. This extraction process identifies and eliminates unnecessary data points while preserving the essential computational functionality, thereby reducing computational workload and resource consumption without significantly impacting accuracy
Solution Approach 2:
The patent applies different properties to different parts of the weight tensor by creating sparsity patterns where certain elements are retained with full precision while others are pruned or approximated. This local differentiation allows the system to maintain high accuracy in critical regions while reducing overall computational burden in less critical regions
2Productivity
If the number of multiplying accumulators is increased to handle computational demands, then computational power improves, but area consumption and power consumption increase
Solution Approach 1:
The patent changes the structural parameters of the weight tensor from dense to sparse format, fundamentally altering how data is stored and processed. This parameter change enables the same computational power to be achieved with fewer physical resources by changing the organization and density of weight data rather than simply adding more hardware units
3Measurement precision
If full precision weight tensors are used, then computational accuracy is maintained, but weight read bandwidth requirements increase
Solution Approach 1:
The patent extracts only the necessary weight elements needed for accurate computation, removing redundant elements that contribute minimally to the final result. This extraction reduces the volume of data that needs to be read from memory while preserving the accuracy required for correct computational outcomes
Data Source
AI summary
A data processing device, a data processing method, and a chip are provided. The data processing device includes a multiplying accumulator. The multiplying accumulator is configured to obtain an input tensor and a sparse weight tensor. The sparse weight tensor is obtained by performing a sparse processing on at least one of a first dimension and a second dimension of an original weight tensor. The multiplying accumulator is further configured to perform a multiply-accumulate (MAC) operation on the sparse weight tensor and the input tensor. The present disclosure enhances the computational power of the data processing device without increasing the number of multiplying accumulators.