DNN Descriptor Scheduling With DMA for Lower Power Inference
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep neural networks (DNNs) require high computational complexity, leading to high power consumption, making them unsuitable for long-duration operations in battery-powered devices and devices with limited power sources, such as smartphones and power-over-Ethernet security cameras, where reduced power consumption is critical.
Innovation Solution
A neural network module that executes descriptors in a layer descriptor list to reduce latency and power consumption by utilizing direct memory access (DMA) engines for memory-to-memory operations, operation descriptors for specific neural network operations, and configuration descriptors for power management, enabling efficient DNN processing while minimizing power usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If general-purpose processors like GPUs are used to provide high computational performance for DNNs, then computational performance is improved, but power consumption increases making them unsuitable for battery-powered devices
Solution Approach 1:
The patent segments the DNN processing function into a dedicated neural network processing unit separate from the general-purpose processor. This neural network processor is specifically designed to handle DNN operations with optimized architecture including specialized memory structures, compute units, and data flow paths, achieving high computational performance while consuming less power than general-purpose processors.
Solution Approach 2:
The patent introduces a descriptor list as an intermediary data structure that enables efficient communication and data transfer between the host processor and the neural network processor. The descriptor list contains pre-configured operation parameters, memory addresses, and computation instructions that allow the neural network processor to execute DNN layers without continuous host processor intervention, reducing overhead and power consumption.
2Measurement precision
If high computational complexity is used to achieve superior DNN performance, then classification accuracy is improved, but power consumption increases making long-duration operations impractical
Solution Approach 1:
The patent enables dynamic adjustment of operational parameters including precision settings, batch sizes, and activation function choices through the descriptor list configuration. This allows the system to optimize the balance between computational accuracy and power consumption based on application requirements, enabling long-duration operations by selecting parameter combinations that achieve sufficient accuracy with lower computational complexity.
3Use of energy by moving object
If ASICs are developed to reduce power consumption while maintaining DNN processing performance, then power efficiency is improved, but device complexity and manufacturing cost increase
Solution Approach 1:
The neural network processor is designed with a universal architecture that can handle multiple DNN layer types and operations through configurable compute units and programmable descriptor lists. This multi-functionality reduces the need for specialized hardware for each operation type, simplifying the overall device architecture while maintaining power efficiency and flexibility.
Data Source
AI summary
A deep neural network (DNN) processor is configured to execute descriptors in layer descriptor lists. The descriptors define instructions for performing a pass of a DNN by the DNN processor. Several types of descriptors can be utilized: memory-to-memory move (M2M) descriptors; operation descriptors; host communication descriptors; configuration descriptors; branch descriptors; and synchronization descriptors. A DMA engine uses M2M descriptors to perform multi-dimensional strided DMA operations. Operation descriptors define the type of operation to be performed by neurons in the DNN processor and the activation function to be used by the neurons. M2M descriptors are buffered separately from operation descriptors and can be executed at soon as possible, subject to explicitly set dependencies. As a result, latency can be reduced and, consequently, the neurons can complete their processing faster. The DNN module can then be powered down earlier than it otherwise would have, thereby saving power.