Convolution Processing Removing Zero Elements
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Neural processing units (NPUs) face inefficiencies in power consumption due to the need to perform convolutions involving zero elements, which result in wasted compute cycles and increased power usage, while also requiring unnecessary computations that do not affect data passing through the Multiply-Accumulate (MAC) unit.
Innovation Solution
The method involves identifying and removing zero elements from both the input feature map (IFM) and the kernel, allowing the MAC unit to perform convolutions only with subsets containing non-zero elements, and decomposing the kernel into sub-kernels to reduce the number of computations, thereby optimizing power consumption and throughput.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If convolutions are performed on complete input data arrays including zero elements, then computational completeness is maintained, but power consumption increases and computational efficiency decreases
Solution Approach 1:
The patent extracts and removes zero-value elements from the input data array before convolution operations. By identifying the positions of zero elements and excluding them from processing, the system performs convolutions only on non-zero elements, thereby reducing unnecessary computational operations and lowering power consumption while maintaining the accuracy of the final output.
Solution Approach 2:
The patent applies partial action by performing convolution operations only on the necessary subsets of data that contain non-zero elements, rather than processing the entire input data array. This selective processing approach reduces the computational workload and energy consumption while still producing the complete and accurate output feature map.
2Loss of energy
If zero elements are processed through MAC units, then complete data flow is maintained, but unnecessary computations increase power consumption
Solution Approach 1:
The patent extracts zero elements from the input data and prevents them from being processed by the MAC units. By using positional data to identify and exclude zero-value positions, the system ensures that MAC units only perform multiplications and accumulations on non-zero elements, thereby eliminating wasteful energy consumption associated with processing zero elements.
3Device complexity
If kernel is decomposed into sub-kernels, then computational complexity is reduced, but data processing steps increase
Solution Approach 1:
The patent segments the kernel into multiple sub-kernels, where each sub-kernel corresponds to a specific subset of the input data. This segmentation allows for parallel processing of different data subsets, reducing the overall computational complexity and enabling more efficient utilization of hardware resources, thereby offsetting the additional processing steps through parallel execution.
Data Source
AI summary
A computer implemented method for performing convolutions between subsets of an input data array and a kernel resulting in subsets of an output data array. The method may include receiving an input data array and using positional data indicating the position of elements of the input data array to determine subsets of the input data array which contains at least one non-zero value data element; performing convolutions between the subsets of the input data array containing at least one non-zero value data element and a kernel to produce output data array subsets; and combining the output data subsets with the positional data to generate output data indicative of a completed output data array.