DNN Module Parallel Processing for Lower Bandwidth and Power
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep neural networks (DNNs) require high computational complexity, leading to high power consumption, making them unsuitable for battery-powered devices and devices with limited power sources, such as smartphones and power-over-Ethernet security cameras, where reduced power consumption is critical.
Innovation Solution
A DNN module utilizing parallel kernel and parallel input processing to decrease bandwidth utilization, reduce power consumption, and improve neuron multiplier stability, along with dynamic clock-gating and power-gating of accumulator memory to achieve power savings.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Power
If general-purpose processors like GPUs are used to provide high computational performance for DNN operations, then computational performance is improved, but power consumption increases making them unsuitable for battery-powered devices
Solution Approach 1:
The patent segments the DNN processing function into a dedicated ASIC module separate from the general-purpose processor. The DNN module contains specialized components including weight buffers, activation buffers, MAC units, and output processors that are specifically designed for neural network operations. This segmentation allows the device to use low-power specialized hardware for DNN tasks while keeping the power-consuming general-purpose processor idle or in low-power mode during these operations.
Solution Approach 2:
The patent introduces a dedicated DNN module as an intermediary between the general-purpose processor and the DNN algorithms. This intermediary module handles all DNN computational tasks, including loading weights from external memory, performing MAC operations, accumulating results, and outputting processed data. The general-purpose processor communicates with this intermediary module through defined interfaces, allowing high-performance DNN processing without requiring the general-purpose processor to operate at high power levels.
2Use of energy by moving object
If ASICs are developed to reduce power consumption while performing DNN processing, then power consumption is reduced, but there is continued need to improve performance and reduce power consumption further
Solution Approach 1:
The patent implements preliminary action by pre-loading weight data into dedicated weight buffers within the DNN module before processing begins. The weight buffers are populated with filter weights from external memory in advance, allowing the MAC units to perform computations without repeated memory access delays. This pre-positioning of data eliminates bottlenecks during actual computation, improving throughput and reducing the time the processor needs to operate at high performance levels.
Solution Approach 2:
The patent introduces parallelism as an additional dimension for processing by implementing multiple MAC units that can simultaneously perform multiply-accumulate operations on different portions of the input data. The architecture includes multiple parallel processing paths with dedicated weight buffers and accumulators for each MAC unit, enabling the system to process multiple feature maps or filter sets concurrently. This parallel execution significantly improves computational performance without proportionally increasing power consumption, as the additional units can be activated only when needed.
3Reliability
If input data is loaded multiple times for processing by multiple kernels, then computational completeness is ensured, but bandwidth utilization increases and power consumption rises
Solution Approach 1:
The patent merges multiple kernel processing operations into a single unified processing pass by maintaining the input data in activation buffers while sequentially applying different kernels. The architecture allows the same input activation data to be processed by multiple MAC units with different weight buffers, eliminating the need to reload the input data for each kernel. The input data remains resident in memory during the sequence of kernel applications, and only the weight data changes between operations.
Solution Approach 2:
The patent ensures continuity of useful action by maintaining input data in activation buffers throughout the entire sequence of kernel processing operations. The architecture is designed so that once input data is loaded into the activation buffers, it remains there and is continuously processed by different MAC units with different weight sets. This eliminates idle memory access cycles and keeps the computational units continuously productive, maximizing the utility of the loaded data and minimizing repeated bandwidth consumption.
Data Source
AI summary
A deep neural network (DNN) module utilizes parallel kernel and parallel input processing to decrease bandwidth utilization, reduce power consumption, improve neuron multiplier stability, and provide other technical benefits. Parallel kernel processing enables the DNN module to load input data only once for processing by multiple kernels. Parallel input processing enables the DNN module to load kernel data only once for processing with multiple input data. The DNN module can implement other power-saving techniques like clock-gating (i.e. removing the clock from) and power-gating (i.e. removing the power from) banks of accumulators based upon usage of the accumulators. For example, individual banks of accumulators can be power-gated when all accumulators in a bank are not in use, and do not store data for a future calculation. Banks of accumulators can also be clock-gated when all accumulators in a bank are not in use, but store data for a future calculation.