Neural Network Feature Map Processing Using Spatial Sparsity
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Neural networks are computationally intensive, particularly in devices with limited resources, leading to high energy consumption, latency, and memory usage, and existing techniques for reducing activation suppression, such as regularization and temporal sparsity, have limitations.
Innovation Solution
Exploit spatial sparsity within feature maps by leveraging correlations between spatially close segments, performing differential and accumulation operations to reduce computations, and utilizing event-driven processing to only process non-zero values.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Use of energy by moving object
If neural network processing is performed using conventional methods, then processing accuracy is maintained, but computational load and energy consumption increase
Solution Approach 1:
The feature map is divided into multiple segments, and processing is performed segment-by-segment rather than on the entire feature map at once. This segmentation allows the system to process only relevant portions of the data, reducing overall computational load and energy consumption while maintaining processing accuracy.
Solution Approach 2:
The system performs partial computations by exploiting spatial sparsity - it computes only the necessary segments of the feature map based on sparsity patterns, rather than performing full computations on all segments. This partial action approach reduces computational load and energy consumption while maintaining the necessary processing accuracy.
2Reliability
If full feature map processing is performed, then processing completeness is ensured, but memory requirements increase
Solution Approach 1:
The feature map processing is segmented into multiple smaller portions that are processed sequentially. By dividing the large feature map into manageable segments, the system reduces the memory required to hold the entire feature map in memory simultaneously, while still ensuring complete processing of all segments for reliability.
Solution Approach 2:
The system performs preliminary analysis to identify sparse regions in the feature map before full processing. This preliminary action allows the system to plan which segments require full processing and which can be handled more efficiently, optimizing memory usage while ensuring processing completeness.
3Productivity
If spatial sparsity exploitation is implemented, then computational efficiency is improved, but processing complexity increases
Solution Approach 1:
The feature map is segmented into multiple regions, and sparsity exploitation is applied to each segment independently. This segmentation approach improves computational efficiency by focusing computations on relevant segments while managing complexity through modular, segment-by-segment processing rather than handling the entire feature map at once.
Data Source
AI summary
Examples described herein relate to neural network processing. Each of a plurality of input feature maps may be processed to obtain respective output feature maps. For each input feature map, a differential feature map is obtained based on differences between corresponding values of a first set of values of the input feature map in respective spatially adjacent segments thereof. A transformation operation is performed on at least a subset of a second set of values of the differential feature map to generate a transformed differential feature map that includes a third set of values. An output feature map is generated and includes a fourth set of values. At least a subset of the fourth set of values is obtained by accumulating respective values of the third set of values with corresponding values of the fourth set of values that were previously generated in the output feature map.