Neural Network Processor Zero-Skipping for Low-Power DNN Inference
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep neural networks (DNNs) require high computational complexity, making general-purpose processors like GPUs unsuitable for long-duration operations in low-power devices such as smartphones and power-over-Ethernet security cameras, where reduced power consumption is critical.
Innovation Solution
A neural network module that skips certain arithmetic operations on input and weight data by determining if entire columns or rows contain zero or near-zero values, allowing for synchronous or asynchronous processing and cooperative neuron assistance to reduce power consumption and improve throughput.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If general-purpose processors like GPUs are used to provide high computational performance for DNN operations, then processing speed and accuracy are improved, but power consumption increases making them unsuitable for battery-powered portable devices
Solution Approach 1:
The processor is divided into multiple processing elements (PEs) that can be independently controlled and powered down. Each PE processes a portion of the DNN computation, allowing the system to activate only the necessary number of PEs based on the current computational workload, thereby reducing overall power consumption while maintaining required processing performance.
Solution Approach 2:
The processor implements dynamic clock gating and power gating mechanisms that periodically activate and deactivate processing elements based on workload demands. During low-computation periods or when processing elements are not needed, their clocks are gated and power is reduced, directly addressing the power consumption issue while preserving computational capability when needed.
2Use of energy by moving object
If ASICs are used to reduce power consumption while performing DNN processing, then energy efficiency is improved, but device complexity and manufacturing difficulty increase
Solution Approach 1:
The processor design uses a universal processing element that can be configured to perform different DNN operations (convolutions, activations, pooling) through programmable control logic rather than dedicated hardwired circuits for each operation. This reduces manufacturing complexity compared to fully custom ASICs while maintaining low power consumption through specialized architecture optimized for DNN workloads.
Solution Approach 2:
The processor implements dynamic reconfiguration capabilities where processing elements can be dynamically allocated and configured based on the specific DNN layer being executed. This flexibility allows the same hardware structure to adapt to different computational requirements without requiring multiple specialized ASIC designs, thereby reducing overall device complexity while maintaining energy efficiency.
3Measurement precision
If all arithmetic operations are performed on input and weight data in DNN processing, then computational accuracy is maintained, but power consumption and processing time increase due to unnecessary operations on zero or near-zero values
Solution Approach 1:
The processor implements sparsity exploitation mechanisms that perform arithmetic operations only on non-zero or significant values in input and weight data. By detecting zero or near-zero elements and skipping computations involving them, the system performs partial action (only necessary computations) rather than exhaustive action on all data elements, significantly reducing power consumption while maintaining computational accuracy through selective processing.
Data Source
AI summary
A deep neural network (“DNN”) module can determine whether processing of certain values in an input buffer or a weight buffer by neurons can be skipped. For example, the DNN module might determine whether neurons can skip the processing of values in entire columns of a neuron buffer. Processing of these values might be skipped if an entire column of an input buffer or a weight buffer are zeros, for example. The DNN module can also determine whether processing of single values in rows of the input buffer or the weight buffer can be skipped (e.g. if the values are zero). Neurons that complete their processing early as a result of skipping operations can assist other neurons with their processing. A combination operation can be performed following the completion of processing that transfers the results of the processing operations performed by a neuron to their correct owner.