DNN Descriptor Execution for Lower Latency and Power Use
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural network processors face challenges in reducing latency and power consumption during deep neural network (DNN) calculations, particularly in battery-powered devices and devices with limited power sources, where high computational complexity and performance are required.
Innovation Solution
A neural network module that executes descriptors in a layer descriptor list, utilizing pre-compiled instructions for forward and backward propagation, including memory-to-memory move, operation, host communication, configuration, branch, and synchronization descriptors, to optimize data transfer and processing, thereby reducing latency and power consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If general-purpose processors like GPUs are used to provide high computational performance for DNN operations, then computational performance is improved, but power consumption increases making them unsuitable for battery-powered devices
Solution Approach 1:
The patent segments the neural network processing into distinct layers and operations, with each layer having its own descriptor and configuration. This allows the processor to execute only the necessary computations for each layer sequentially, improving energy efficiency by avoiding unnecessary computations while maintaining high performance through specialized hardware acceleration for each segment.
Solution Approach 2:
The patent implements preliminary action through pre-compiling neural network configurations into layer descriptor lists before execution. The host system prepares the entire DNN configuration, including all layer parameters and operations, in advance and transfers them to the neural network processor in a single operation. This eliminates runtime configuration overhead and allows the processor to enter efficient execution mode immediately, reducing overall power consumption while maintaining high computational throughput.
2Use of energy by moving object
If descriptor lists are transferred from host memory to neural network processor memory in small increments, then power consumption during transfer is reduced, but processing latency increases
Solution Approach 1:
The patent implements periodic action by transferring descriptor lists in structured batches corresponding to neural network layers. Instead of continuous or single-descriptor transfers, the system periodically transfers groups of descriptors representing complete layers or operation sequences. This rhythmic transfer pattern allows the host and processor to synchronize efficiently, reducing peak power consumption during transfers while maintaining overall processing throughput by keeping the processor continuously engaged with incoming descriptor batches.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A deep neural network (DNN) processor is configured to execute descriptors in layer descriptor lists. The descriptors define instructions for performing a pass of a DNN by the DNN processor. Several types of descriptors can be utilized: memory-to-memory move (M2M) descriptors; operation descriptors; host communication descriptors; configuration descriptors; branch descriptors; and synchronization descriptors. A DMA engine uses M2M descriptors to perform multi-dimensional strided DMA operations. Operation descriptors define the type of operation to be performed by neurons in the DNN processor and the activation function to be used by the neurons. M2M descriptors are buffered separately from operation descriptors and can be executed at soon as possible, subject to explicitly set dependencies. As a result, latency can be reduced and, consequently, neurons can complete their processing faster. The DNN module can then be powered down earlier than it otherwise would have, thereby saving power.