Neural Network Processing with Zero-Value Operation Skipping
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current neural network processors face challenges in reducing power consumption, particularly in battery-powered devices and non-battery-powered devices with limited power sources, where continuous deep neural network tasks require high efficiency to extend battery life or operate effectively.
Innovation Solution
A neural network module that skips certain arithmetic operations on input data and weight data by identifying zero or near-zero values, allowing for full-column or asynchronous operation skipping, thereby improving throughput and reducing power consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If general-purpose processors like GPUs are used to provide high computational performance for DNNs, then processing speed and accuracy are improved, but power consumption increases making them unsuitable for battery-powered devices
Solution Approach 1:
The patent extracts and processes only the non-zero elements from sparse input data and weight matrices, separating the useful computational work from the redundant zero-value operations. This is achieved through identifying sparse regions and performing computations only on the extracted non-zero elements, thereby reducing power consumption while maintaining computational accuracy for DNN inference tasks
Solution Approach 2:
The patent applies partial action by performing computational operations only on the necessary non-zero portions of data rather than processing entire matrices. The system identifies sparse regions and executes arithmetic operations exclusively on non-zero elements, using partial computation to achieve the same functional outcome with reduced energy expenditure, making DNN processing viable for battery-powered devices
2Use of energy by moving object
If ASICs are developed to reduce power consumption for DNN processing, then energy efficiency is improved, but device complexity and manufacturing difficulty increase
Solution Approach 1:
The patent implements a universal processing architecture that can handle both dense and sparse DNN workloads through the same hardware pipeline. The system includes configurable components that adapt to different data formats and computational requirements, allowing a single device to serve multiple DNN processing needs without requiring specialized ASIC designs for each application, thereby reducing manufacturing complexity while maintaining energy efficiency
Solution Approach 2:
The patent employs dynamic processing modes that can switch between different computational strategies based on input data characteristics. The system dynamically identifies sparse regions and adjusts its processing approach accordingly, enabling flexible adaptation to varying workload requirements without fixed hardware configurations, thus reducing device complexity compared to dedicated ASIC implementations
3Measurement precision
If all arithmetic operations are performed on input data and weight data to ensure complete processing, then processing accuracy is maintained, but power consumption and processing time increase
Solution Approach 1:
The patent converts the harmful effect of sparse zero-value operations (which consume power but contribute nothing to the result) into a benefit by using the sparsity pattern itself as a guide for optimization. The system identifies and exploits the sparse structure of DNN data to skip unnecessary computations, transforming what was previously a waste of resources into a mechanism for energy savings while preserving complete processing accuracy for non-zero elements
Solution Approach 2:
The patent implements skipping of arithmetic operations on zero-value elements by identifying sparse regions in advance and bypassing redundant computations entirely. The system rushes through the data processing by jumping over zero portions and directly computing only the necessary non-zero contributions, thereby maintaining accurate results while significantly reducing the time and energy required for DNN processing
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A deep neural network ("DNN") module can determine whether processing of certain values in an input buffer or a weight buffer by neurons can be skipped. For example, the DNN module might determine whether neurons can skip the processing of values in entire columns of a neuron buffer. Processing of these values might be skipped if an entire column of an input buffer or a weight buffer are zeros, for example. The DNN module can also determine whether processing of single values in rows of the input buffer or the weight buffer can be skipped (e.g. if the values are zero). Neurons that complete their processing early as a result of skipping operations can assist other neurons with their processing. A combination operation can be performed following the completion of processing that transfers the results of the processing operations performed by a neuron to their correct owner.