Compressed Activation Tensor for Deep Neural Network Efficiency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep neural networks (DNNs) face high computational complexity and resource wastage due to processing zero activations in MAC operations, which hampers efficiency in tasks like image recognition and natural language processing.
Innovation Solution
Exploiting activation sparsity by generating a compressed activation tensor with non-zero activations, which reduces the number of columns and processes them efficiently with MAC hardware, thereby minimizing resource wastage and improving computational efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If deep neural networks process all activations including zeros in MAC operations, then computational completeness is maintained, but computational complexity and resource wastage increase
Solution Approach 1:
The patent extracts and removes zero activations from the activation tensor before MAC operations. By generating a compressed activation tensor that contains only non-zero activations, the system eliminates unnecessary computations while preserving the essential computational information needed for accurate neural network processing.
Solution Approach 2:
The patent changes the parameter of activation sparsity by transforming the full activation tensor into a compressed version with reduced dimensionality. This parameter change allows the system to process fewer elements while maintaining computational accuracy, thereby improving efficiency.
2Power
If deep neural networks process zero activations in MAC operations, then computational completeness is maintained, but energy consumption increases
Solution Approach 1:
The patent extracts and removes zero activations from the activation tensor before MAC operations. By generating a compressed activation tensor that contains only non-zero activations, the system eliminates unnecessary computations while preserving the essential computational information needed for accurate neural network processing.
Solution Approach 2:
The patent changes the parameter of activation sparsity by transforming the full activation tensor into a compressed version with reduced dimensionality. This parameter change allows the system to process fewer elements while maintaining computational accuracy, thereby improving efficiency.
3Quantity of substance
If deep neural networks use standard activation tensors, then data structure simplicity is maintained, but memory bandwidth and storage requirements increase
Solution Approach 1:
The patent extracts and removes zero activations from the activation tensor before MAC operations. By generating a compressed activation tensor that contains only non-zero activations, the system eliminates unnecessary computations while preserving the essential computational information needed for accurate neural network processing.
Solution Approach 2:
The patent applies dimensionality change by compressing the activation tensor from its original high-dimensional form to a lower-dimensional compressed version. This reduces the number of columns while maintaining the essential information, thereby reducing memory bandwidth requirements and storage needs.
Data Source
Figure 1
Figure 2A~2C
Figure 2D
AI summary
A method of exploiting activation sparsity in deep neural networks is described. The method includes retrieving an activation tensor and a weight tensor where the activation tensor is a sparse activation tensor. The method also includes generating a compressed activation tensor comprising non-zero activations of the activation tensor, where the compressed activation tensor has fewer columns than the activation tensor. The method further includes processing the compressed activation tensor and the weight tensor to generate an output tensor.