CNN Kernel Sparsity Using Inter-Kernel Weight Broadcasting
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep learning using Convolutional Neural Networks (CNNs) involves significant computational costs due to the large number of multiplication and accumulation operations, which can be reduced by exploiting sparsity in input feature maps and kernels, but existing pruning methods can lead to accuracy degradation.
Innovation Solution
A method and system that generate encoded Similar or Identical Inter-Kernel Weight (S/I-IKW) streams to convert similar and identical weights in non-pivot kernels to zero, using a Neural Processing Unit (NPU) to broadcast values and perform convolution operations, thereby introducing sparsity and reducing computational costs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Use of energy by moving object
If pruning method is used to introduce sparsity in kernel, then computational costs are reduced, but inference accuracy of CNN is degraded
Solution Approach 1:
The patent merges the functionality of multiple kernels by identifying and exploiting identical weight patterns across kernels. Instead of treating each kernel independently, the system combines their computational paths, allowing shared weight calculations and reducing redundant operations while preserving the collective predictive power of all kernels.
Solution Approach 2:
The patent makes kernel weights universal by identifying identical weights that serve multiple kernels simultaneously. A single weight calculation can be reused across multiple kernels through the identified identical patterns, allowing one computational result to fulfill multiple functions and reduce overall computational burden.
2Productivity
If millions or billions of multiplication and accumulation operations are performed, then CNN computation is completed, but computational costs and power consumption increase
Solution Approach 1:
The patent extracts identical weight patterns from multiple kernels and separates them into a shared computation path. By taking out the common identical weights and handling them separately through broadcasting mechanisms, the system eliminates redundant multiplication operations while maintaining the necessary accumulation operations for accurate inference.
Solution Approach 2:
The patent performs preliminary identification of identical weight patterns across kernels before the main convolution operation. By pre-processing the kernel weights to identify and mark identical patterns, the system prepares the computational path to exploit these patterns during inference, reducing the number of actual multiplication operations needed.
3Use of energy by moving object
If sparsity is introduced in IFM and kernel, then computational costs are reduced, but pruning impact accuracy
Solution Approach 1:
The patent introduces dynamic sparsity exploitation by identifying identical weight patterns and adaptively broadcasting their results. Rather than static pruning, the system dynamically determines which weights are identical across kernels and adjusts the computation path accordingly, maintaining accuracy for non-identical weights while exploiting sparsity for identical ones.
Data Source
AI summary
A processor-implemented method for generating Output Feature Map (OFM) channels using a Convolutional Neural Network (CNN), include a plurality of kernels, includes generating at least one encoded Similar or Identical Inter-Kernel Weight (S/I-IKW) stream, converting, similar and identical weights in the at least one non-pivot kernel to zero to introduce sparsity into the at least one non-pivot kernel, broadcasting at least one value to the at least one non-pivot kernel, and generating at least one OFM channel by accumulating an at least one previous OFM value with any one or any combination of any two or more of a convolution of non-zero weights of the pivot kernel and pixels of the Input Feature Map (IFM), the at least one broadcasted value, and a convolution of non-zero weights of the at least one non-pivot kernel and pixels of the IFM.