DNN Layer Fencing and Dependency Control for Low-Power Inference
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current general-purpose processors, such as GPUs, are unsuitable for long-duration DNN tasks in low-power devices like smartphones and AR/VR devices due to high power consumption, and existing ASICs do not adequately address the need for reduced power consumption and improved performance for DNN processing.
Innovation Solution
A neural network module that executes layer descriptors in a layer descriptor list to optimize the flow of descriptors, reducing latency and enabling faster processing, which allows the module to be powered down earlier, thereby saving power.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If general-purpose processors like GPUs are used to provide high computational performance for DNNs, then processing speed is improved, but power consumption increases
Solution Approach 1:
The patent segments the DNN processing workload into distinct operational phases (forward propagation, backward propagation, weight updates) and implements specialized circuitry for each phase. This allows the processor to activate only the necessary computational units for each task, reducing overall power consumption while maintaining high processing speed for each specific operation.
Solution Approach 2:
The processor implements dynamic voltage and frequency scaling (DVFS) that adjusts operational parameters based on the current DNN layer being processed. Computationally intensive layers receive higher power and frequency, while simpler layers operate at lower power states, optimizing the balance between processing speed and power consumption in real-time.
2Use of energy by moving object
If ASICs are developed to reduce power consumption for DNN processing, then power efficiency is improved, but device complexity increases
Solution Approach 1:
The patent designs a universal DNN processor architecture that can execute multiple types of neural network operations (convolution, fully connected, pooling, normalization) through a single configurable hardware platform. This multi-functionality reduces the need for multiple specialized ASICs, thereby lowering overall device complexity while maintaining power efficiency.
Solution Approach 2:
The processor employs configurable parameters that can be adjusted at runtime to match different DNN layer requirements. By changing operational parameters rather than hardware configuration, the system achieves specialization for different tasks without increasing physical device complexity, maintaining power efficiency across various workloads.
3Productivity
If DNN processing is optimized for speed, then productivity is improved, but power consumption increases
Solution Approach 1:
The patent implements periodic activation of computational units based on the processing requirements of different DNN layers. Instead of continuous operation, processing units are activated in periodic bursts synchronized with data flow through the network, achieving high processing speed during active periods while consuming minimal power during idle periods.
Solution Approach 2:
The processor maintains continuous data flow through the pipeline, ensuring that once computation begins, it proceeds without interruption to maximize processing speed. Simultaneously, the system quickly transitions to low-power states between consecutive DNN inference tasks, minimizing energy consumption during transitions while maintaining high productivity during active computation.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A deep neural network (DNN) processor is configured to execute layer descriptors in layer descriptor lists. The descriptors define instructions for performing a forward pass of a DNN by the DNN processor. The layer descriptors can also be utilized to manage the flow of descriptors through the DNN module. For example, layer descriptors can define dependencies upon other descriptors. Descriptors defining a dependency will not execute until the descriptors upon which they are dependent have completed. Layer descriptors can also define a "fence," or barrier, function that can be used to prevent the processing of upstream layer descriptors until the processing of all downstream layer descriptors is complete. The fence bit guarantees that there are no other layer descriptors in the DNN processing pipeline before the layer descriptor that has the fence to be asserted is processed.