DNN Fabric Interface Bandwidth Throttling for Lower-Power Throughput
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current deep neural network (DNN) processing units face challenges in achieving optimal performance and reducing power consumption, particularly in devices where low power consumption is critical, such as battery-powered devices and power-over-Ethernet security cameras, due to inefficiencies in memory and processor management.
Innovation Solution
Implementing bandwidth throttling mechanisms, including specifying cycles to wait between transactions and transaction count limiters, to optimize bandwidth usage and reduce power consumption by ensuring uniform utility of processing units in DNN modules.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If general-purpose processors like GPUs are used to provide high computational performance for DNN operations, then processing performance is improved, but power consumption increases making them unsuitable for battery-powered portable devices
Solution Approach 1:
The patent changes the operational parameters of the DNN processing system by implementing dynamic bandwidth throttling mechanisms that adjust fabric interface transaction rates. This allows the system to operate at optimal performance levels while reducing unnecessary data transfers and associated power consumption, particularly in ASIC-based DNN processors where fixed high-performance modes consume excessive power in portable devices
2Stability of the object's composition
If memory and processor management follow current practices to balance resource requests, then system stability is maintained, but processing performance and power consumption optimization is insufficient
Solution Approach 1:
The patent implements feedback mechanisms where the DNN processor monitors fabric interface bandwidth utilization and dynamically adjusts throttling parameters based on actual processing needs. This feedback loop ensures system stability is maintained while optimizing performance by preventing both over-provisioning (wasted power) and under-provisioning (performance bottlenecks) of memory and processor resources
3Use of energy by moving object
If bandwidth throttling is applied to limit average and peak bandwidth usage, then power consumption is reduced, but processing performance may be degraded
Solution Approach 1:
The patent applies dynamic bandwidth throttling where the throttling parameters are not fixed but adapt based on the current operational state of the DNN processor. The system dynamically adjusts the number of cycles to wait between fabric transactions and the number of in-flight transactions based on real-time bandwidth utilization, ensuring optimal balance between power consumption and processing performance at different operational phases
Data Source
AI summary
An exemplary computing environment having a DNN module can maintain one or more bandwidth throttling mechanisms. Illustratively, a first throttling mechanism can specify the number of cycles to wait between transactions on a cooperating fabric component (e.g., data bus). Illustratively, a second throttling mechanism can be a transaction count limiter that operatively sets a threshold of a number of transactions to be processed during a given transaction sequence and limits the number of transactions such as multiple transactions in flight to not exceed the set threshold. In an illustrative operation, in executing these two exemplary calculated throttling parameters, the average bandwidth usage and the peak bandwidth usage can be limited. Operatively, with this fabric bandwidth control, the processing units of the DNN are optimized to process data across each transaction cycle resulting in enhanced processing and lower power consumption.